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propositions and that now as in time past they run about 
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union of them will neve’^ cease and is not now beginning 
but is as I believe an ev rlasting quality of propositions 
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have no chance of escaping him if an interpreter could only 
be found 


Socrates (The dialogues of Plato 
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SYNOPSIS 


Poonacha P G Ph D 
Department of Elec rical Engineering 
Irdian Institute of Technology Kanpur 
October 1986 

FINITE DECONVOLUTION DATA RECOVERY SCHEMES FOR 
BLOCK DATA COMMUNICATION OVER DISPERSIVE CHANNELS 


In this thesis we study several data recovery (DR) schemes 
for block data transmission (BDT) over finite impulse response 
channels in the presence of noise In block data transmission 
due to Clark the input sequence is partitioned into blocks 
which are separated by guard zones of appropriate duration such 
that the channel output blocks do not overlap Since the 
channel is assumed to be linear its output can be represented 
as convolution of the input data values and the channel impulse 
response The operation of recovering data from the nonover- 
lapping blocks at the receiver is then essentiall/ that of 
finite deconvolution in the presence of noise 

We formulate the deconvolution problem as minimization of 
a banded Toeplitz quadratic function with data as unknown varia- 
bles When the unknown variables are real minimization of the 
quadratic function is straightforward However when the varia- 
bles take values from a finite alphabet as in the case of data 
transmission it is dif-^icult to provide computationally attra- 
ctive solutions The solution to the deconvolution problem is 


optimal or suboptimal depending on the assumptions made on the 
unknown variables and the strategy adopted for solving the pro- 
blem 

We study maximum likelihood decision feedback and least 
squares deconvolution methods in detail Among these schemes 
maximum likelihood scheme is an optimal deconvolution scheme 
while decision feedback and least square deconvolution schemes 
are suboptimal Our main concern throughout is to understand 
various theoretical and computational issues involved in 
developing DR scheme based on these methods 

Salient features of the DR schemes based on the above- 
mentioned deconvolution methods are briefly described in the 
following paragraphs 

1 Maximum Likelihood Deconvolution Scheme 

fie first derive a maximum likelihood (ML) recursive 
deconvolution algorithm (ML-RDA) based on Mitten s lemma which 
provides conditions for optimal decomposition of multistage 
decision processes This algorithm performs essentially the 
job of ML sequence estimation based on Vitorbi algorithm 
However the derivation of the algorithm does not require the 
use of state representation and explicit invocation of Bellman 
principle of optimality which are essential in deriving Viterbi 
algorithm The approach taken provides better insight into the 
ML deconvolution problem compared to the conventional VA appli- 
cation We have shown that if tho dat v lues are assumed to 



be real variables the recursive algorithm reduces to the 
familiar Gaussian elimin tion algorithm for solving a set of 
linear equations We can therefore view ML deconvolution al- 
gorithm as a generalized Gaussian elimination algorithm for 
minimizing banded Toeplitz quadratic objective functions 

In the case of binary data v/o show that the ML RDA 
reduces to a special case of tho algorithm given by Hammer 
et al for minimizing psoudo Bool an functions We further 
show that in the binary case the problem of reducing storage 
and computational complexity of ML RDA involves determination 
of minimal sum of products form of Boolean functions of 
several variables Therefore fast algorithm available for 
such purpose can be used to speed up computations and to 
reduce storage requirements Finally we show that the recur- 
sive algorithm derived for minimizing pseudo Boolean function;^ 
Can also be used for ML-decoding of binary block cod-^s Use- 
fulness of tho method is illustrated with the help of examples 
However much remains to bo done in this direction 

2 Decision Feedback Deconvolution Schemes 

We next consider the possibility of incorporating decision 
feedback (DF) in DR schemes for BDT with the objective of 
achieving good performance without going in for f\L-deconvolu- 
tion which may not be economical in some practical situations 


DF~DR schemes consist of a forward filter for minimizing 
the effect of future symbols a nonlinear decision device and 
a backward filter for eliminating the effect of past symbols 
This scheme may be viewed as Gaussian elimination method of 
solving linear equations using forward elimination and backward 
substitution with a nonlinear decision device interposed bet 
ween forward elimination and back ubstitution tages 

For deriving DF~DR sch mes we first minimize the banded 
Toeplitz quadratic function assuming that tho data values are 
real This leads to the solution of banded Toeplitz system of 
linear equations Wo then derive throe DF-DR chemes for BDT 
The first scheme is based on Cholesky decomposition of positive 
definite matrices This scheme however does not take noise 
V nance into account The second DF-DR scheme takes noise 
variance into account and is based on minimum mean square error 
L criterion The third DF-DR scheme is based on minimum property 
of Fourier expansions over finite dimensional HilLert spaces 
This scheme is useful when the channel autocorrelation matrix 
IS illconditioned and when no knowledge of the noise variance 
is available 

Finally we derive a DF-DR scheme by modifying Austin s 
detection theory approach We show that the modified Austin s 
method can also be viewed as Gaussian elimination method with a 
nonlinear decision device 



Least Squares Deconvolution Schemes-Direct App oach 

then undertake a ritical stuay o 1 ast uares (LS) 
deconvolution using discrete Fourier transforn (DFT) techniques 
The main f ature of our stuoy is in providing a general treat- 
m nt to the LS~decon /olution probl m using the notion of square 
completions of the channel onvolution ma rix V/e show that 
ooth LS solution and the con/ ntional DFT apoioach are particu- 
lar cases of square completions of the chann 1 convolution 
ftatrix Moreover DFT approach to the d convolution probl m is 
a particular case of k-Circclant completion of the channel con- 
voluiion matrix Based on k-Lirculant completions we develop 
two data recovery schemes which are easily implement able and 
ho^ve redu ed computational complexity 

Although DFT based deconvolution is widely used condi- 
tions under which such techniques provide optimal solution to 
d'=‘Convolution problem according to some criterion do not seem 
to have been explicitly m ntioned Me investigate condmons on 
k Circulant conoletions which provide Lo solution to the decon- 
volution problem Using the framework of quare completion of 
matrices we obtain conditions on square completion that pro- 
vide Moore-Penrose inverse of a full rank matrix Such cotrple 
tions are called orthogonal completions Then it is s nown that 
orthogonal k-Circulan romo3 tions are orthogonal k-Circulant 
na'^iices Using this resuJ t v;e show that for all practic 1 



ases of interest LS solution to the deconvolution problem can 
not be obtained via direct inversion of k Circui nt completions 

4 Least Squares Deconvolution Scheme - Iterative Approach 

Finally we con ider an ite atave d con /olution scheme 
using Steepest Descent Algorithm (SDA) One of the purposes of 
this study IS to examine tie obs'^rv tion of several authors 
working with fixed st^-p SDA in the DFT domain that the DFT 
domain iteration scheme converges faster thin the sample 
domain scheme For this purpose deriv SD^ in the DFT 
domain from three dif rent view points We show that the DFT 
domain scheme does not alter the conv rgence rate it only 
helps in reducing computational complexity at each iteration 
Ho\/ever it is shown that k-Circulant comole cions can be used 
to obtain good estimates of the convergence parameter in the 
case of some examples These es tinatod values can be used 
either in the sample domain or in the DFT domin to accelerate 
convergence 

Since the iror performance of each data recovery scheme 
strongly depends on the particular channel used we give an 
account of good channels for the ourpose of comoaring the per- 
formance of various DR sch mes Two criteria for good channels 
are considered The first criterion provide channels having 
maximum minimum distanc (f lu) called \ru O-channels These 
channels re used as benchmarks for comparing the performance 
of various DR scheme used on actual channels 



Determination of MI ID^ch annals however is a difficult 
problem Therefore we consider the second class of good 
channels called maximum distance channels (i iD-channels) which 
maximize tie total distance A procedure is given for obtaining 
Mi\/lD~channels via MD-channel characterization For the purpose 
of comparing actual channels with MflD-char ncls we also deielop 
a recursive algorithm for compuiing the minimum distance of a 
given channel It is shown that some of the MiiD and MD channels 
are partial response channels 

The performance of some of the data recovery schemes dis- 
cussed earlier has been evaluated using computer simulation 
Performance curves showing the variation of probability of error 

with signal~to~noise ratio are given for a wide range of block 
lengths and typical channel examples 

The simulation stud> brings out the following merits of SDT 
over continuous serial data transmission (C nr) 

1) If decision feedback idea is used to improve the perfor- 
mance of BDT systems error propagation is limited to almost a 
block length in the worst case whereas in the case of DF-DR 
schemes for CSDT error propagation can extend over much longer 
duration Since in the rarge of n dium signal-to-noise ratios 
the error propagation is a serious probl n with DT DR schemes 
for CoDT systems BDT with DF DR schemes are to b preferred in 
such applications 



2) '/ith an appropriate choice of the block length BDT can 
be used to provide better trade-off betw en perfornance proce- 
ssing complexitv and data rate 

A drawback of BDT is that the effecti/e data rate over the 
channel is less compared to CSDT for a given source rate The 
loss in data rate may be compensated by transmitting data over 
the channel at a higher rate This result in increased inter- 
synbol interference among the block samples Our simulation 
studies how vei indicate that if the block length is properly 
chosen the degradation in perfornance of DR schemes for BDT 
with increased intersymbol inteifcrence will not be significant 



CHAPTcR 1 


INTRODUCTION 


In this ho 3 s we study several data recovery (DR) scheme 
for Diocl data tra'^ inssion over linear time dispe’^’sive channels 
in the pr scnce of noise In block data transmission tBDT) the 
input data sequence is pamtioned into blocks which are separa 
ted oy guard zone o aporooriabe duration such that the channel 
output blocks do not overlap ince the haniel is assumed to 
be linear its outputs can be I'epresented as convolution of the 
input data values and th ch rnol impulse response The opera 
tion of recovering data from the nonoverlapoing blocks at the 
channel output is then essencially that o finite deconvolution 
in the presence of noise thai j given the channel impulse 
response nd the nonoverlapping hannel output blocks which 
have been corrupted by noise tho probi m is to recover the data 
at the receiver ond in some optimum manner 

Ve formulate the deconvolution proolem as minimization of 
a banded Toeplitz quadratic function /ith data as unknown varia 
bios When th unknown variables are r al minimization of the 
quadratic function is straightforward however when the varia- 
blcs take values fron a finite alphabet as in the case of data 
tr nsmission it as difficult to provide romputi lonally attra- 
ctive solutions The olution to tho d con\olution problem is 
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optimal or suooptmal depending on the assumptions made on the 
unknovm variables and the trategy oopted for solving the 
problem 

Vife study maximum likelihood a cision f elback and least 
squares decon /olution methods n detail Ationg chese schemes 
maximum likelihood ( tL) scheme is an optimal deconvolution 
scheme wnile decision feedback (DF) and least squares (LS) 
deconvolution schemes a suboptimal Our ma n concern through- 
out IS to understand va lous theoretical and computational 
: ssuos involved in dev'^loping DR schemes based on those methods 

1 1 MOTIVATION 

Design and evaluation of UR sch mes for long strings of 
data transmitted over time Jispersiv chann Is in the presence 
of roise IS an mpoitan topic of study in the area of digital 
communication Development of v nous data transmission and 
recovery che np which date back to the pion cring work of 
Nyqui t [l] and which continue to draw a great deal of atten- 
tion oven today has been primarily lotivated hy a de ire to 
increase the d ta rate Thi is refl cted in the design of 
most of the available data transmission and rccovory schemes 
In this connection continuous serial data txansrassion (CSDT) 
happens to be the mo t vAdely used data transmission scheme in 
;hich a continuous string of serial d a is brans aittod over 
the channel 
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In thr case of data tr nsnission over time disp rsive 
channels due to int r ymbol interfe ence (ISI) the signal 
sample at the receiver input b come highly correlated There- 
for imole nenti tion of optimal Dr schents for CSDT system 

o on s imoractical as the/ require very larg obs rvation 
int rvals which m ans prohibitively larg delays and processing 
recfuire aents One method of circumv nting th se problems 
wh ch has 3 ceiv d considerable cittention is to nake d-'cisions 
based on truncated observation intervals This strategy 
reduces delay and processing time at the expense of increased 
orobabil of erior The trade-off however apoears to be 
one sid d rom the point of viev^ of the following thioe major 
Jeman say good data transnission sy t m i eyp^^ctod to 
satisfy 

D1 Mini lum probability of error The piobability of a 
de pct d da a symbol L ing in rror must be ninimum 

D2 Minimum proe ssing tim Compu La b3 onal conolexity of 
the data recovery scheme nust be mini num no the sch me must be 
easily implc i ntablr 

D3 Maximun dat rate It must be oossible to transmit 
nata v lues on aft r another with minimum tjno gap between 
adj ac nt synbols 

The orego ng ciscussion highlights the lart th t there is 
s ill n v.a or in /es tigating alternate \/ays of oata transmission 
v^hich 1x11 holp in striking a bot cr compromise between D1 D2 
and D3 



An alternate method of r'=‘ducing delay and proc'^ sing 
r quirement vuthout sacriiyinc error peif or nance i to reduce 
he observ tion inter /al by aDpropriatel> ch ging the data 
tran miss on fornat Thi may be achieved by u ing BDT format 
studied bv Clark [2] - [4] tudy various da'* a recovery 

schenes for BDT vith tne following objectives in m rd 

1 \ better under tand ng of t}' e possible trade-off between 
performance processing comolexity and data transmission 
rate in the context of BDT 

2 A better appreciation of the merit anl dem iits of the 
existing DR schemes i or GSuT 

3 To criticall/ anal> e the advantages of BUT in situations 
where prooability o ciror performance ot the transmission 
system is more imporlant than achieving maximum data rate 
at the cost of error pcrfoimance as in the c e of CSDT 

1 2 REVIEW OF LITbRATURE 

To place our study in a proper perspective wc hall first 
review the exi ting literature on DR schemes for C^jDT and BDT 
Our int rest is not to give a detailed account of the histori 
cal ae/elopmont of v ra ous DR schemes Inste d wc wish to 
highlight some of the key developnont with the help of rol 
vant literature It may be pointed at the outset that most of 
the work is in connection with CSDT ref rences on BDT axe 
very few Wo review the literature on CSDT first and consider 
BDT later 


An alternaip> method of reducing delay and proc asing 
requirements without sacriiyinci error porf or nance i to reduce 
he observ tion inter /al by aoprop-riately ch ging the data 
tran miss on fornat Thi may be achieved by u ing BDT format 
studied bv Clark [2] ~ [4] }e tudy vinous data recovery 
scheries for SDT -^ith tne following objectives in m nd 

1 ^ b tter under tend ng of iJ e possible trade-offs between 
performance processing comolexit/ and data transmission 
rate in the context of 3Dr 

2 A better appreciation of the merit an 1 dem^^rits of the 
existing DR schem s lor GSuT 

3 To critically anal> e the advantages of 3JJT in situations 
where procability o error performance of the transmission 
system is more imporiant than achieving maximum data rate 
at the CO t of error performance as in the c se of CSDT 

1 2 REVIEW OF LITfcRATUPE 

To place our study in a proper perspective wc hall first 
review the exi ting literature on DR schemes for C*Dr and BDT 
Our int rest is not to give a detailed account of the histori 
cal de/elopment of v nous DR schemes Instc d we wish to 
highlight ome of the key developments with the help of role 
vant litei aturo It may be pointed at the outset that most of 
the work is in onnection with CSDT references on BDT are 
very lew Wo review the literature on CSDT first and considfP 
BDT la tor 
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Transmission 

The literature on DR schemes for CSDT in t e presence of 
ISI and noise is ri h wixh a wid variety of novel pproach s 
and related theoietical acvelopments A number of well written 
books are available on this topic i xts by Bennet and 
Davey [5] Lucl y 6alz and W Idon [6] and Proakis [v] provide 
excellent coverage of most of the important topics in the 
area of PoOT longwith long lists of references the text book 
by Clark [4] oroviaes wide coverage to both CSDT and BDT The 
collection of papers eaitcd by Franks [s] on data communication 
provides at a place tw nty four important works on CSDT along~- 
vath informative and well written long ditorial comments 
Important dev lopnents that took place 3n th'^ do i in of CSDT 
during 1968 73 are discussed with extensive bibliography in 
a survey article by Lucky [9] Most o the r oorr d data 
r covery schemes for CoDT are either some variant of one of th 
following thro schemes or their combinations 

1 Linear Data Recoveiy uchem s 

Th first and the mosr simple one is th linear DR scheme 
consi ting of a ma ch d filter mat hod to the channel folJowed 
by a trans/e sal filter whose output xs ampl d and used in 
making decisions First s st matic study in this connection 
app ars to b du to Tufts [lO] [ll] Extending th cla sical 
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work of Nyquist [l] Tufts [lO] has den /ed an optimal linear 
receiver structure using iinimum mear square error (f SB) cri 
tenon As an extension of this work Smith [12] has obtained 
explicit solution to the optimum receiv r when the transmitted 
wavefoTm is subjected to an average power constraint The cri 
tenon optimization howe /er remains same as before 

Deviating considerably from the previous approaches and 
u ing detection theory point of view Austin [13] has shown 
that linear receiver can be derived on the assumption that the 
data values are indep ndenl GdUssian random variables The 
resulting transv^ersal filter in general is of infinite length 
However for practical purposes only finit length filters are 
employed and the tap coefficients are optimized by minimizing 
the MSB between the filter output at the sampling instant and 
the actual data value Due to th^ir simplicity finite transv'=‘r~ 
snl filters have b en us d extensively for the p irpose of auto 
matic channel qu^lization [14J”[22] as well as for adaotive 
equalization [23]'-[28] 

2 Decision Feedback Data R covery Scheir s 

The second DR scheme known as decision fe dback equalizer 
(DFt.) [13] [29] consists of a matched filter sampl r forward 

filtfr backward f Iter and a nonlinear decision device DFE 
makes use o-^ p t decisions to improv p rfoT lance Howev r as 
the actual decisions are always orono to rror ( xcopt possibly 
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at high sigrial-to-noise ratio (SinI-i)) DFc auffers from the 
problem of error propagation DFIl structure can be derived on 
the assumption that the past symbols aie erroi iiee and that 
the future symbols are independent Gaussian random variables 
Usirg this assumption Austin [13] has derived he structure of 
the DFE The resulting forward filter is in general of infi- 
nite length Hovever a in the case o linear L)R schemes it 
IS common practice to use only finite length and the tap 
CO fficients are optimiz d using \iSE criterion DFC has since 
been studied from arious viewpoints [30]-[37] 

3 Maximum Likelihood Data Kecovery Scheme 

The third DR scheme kiovn as maximum likelihood DR scheme 
IS due to Forney [38] He employed Viterbi algorithm (VA) [39]- 
[42] to alleviate th»^ co nputa tional complexity associated with 
ML sequence estimation ot data Th Ml sequence stimator 
minimi os the probability o-(' soquenc eiror when all the 
s quences are qually likel/ We yyish to note here that an 
algorithm similar to VA has been dev loped by Kobayashi [43"' 
for ML decoding of a particular typo of PAIk signnl before the 
publication oi the Forn y s now classical work [3o] 

For ML s quence estima lor the Vr\ requires ntiie obsorva 
tD on values For long da t s leams such a sch m therefore 
becomes un tractive bo h from the point of view of computa- 
tional corplcxity and storag-^ r quiromcnts To make the VA 
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useful decisions ar mad after a fixed numb r of observations 
uccom s av ilaol this nuno r is fixed empirically on th basis 
of channel nemory with the ssumption that d gradation due o 
such premature decision is n jl3G ble [3^^] However since the 
perforn nee oegradation strorgl d pends on the channel and 
noiso such an assumption n cds c reful c^aninjtion 

Aft r Forno/ s v;ork all for s on ML DR schemes soom to 
have been dir c ced towards furth^'^' reducing the conolexity of 
\/A [ ^]-[52j Specific liy since the comp itational complexity 
of the Vft grov; xponen ti Uy wit^ the channel memo’"/ efforts 
hav been nad to shorten ic channel impulse response duration 
by using prefiltc rs 

12 2 Data Pecovery chern s for Block D a Transmission 

Clar< [2] [4j has inves ig t d a number of adaptive dete- 
ction orocGs for BDT syst n In his work the 6Di has been 

referred to as irans nission o o thogonal group of signal 
elen nts by v rtuc of h fact that at the channel output such 
groups oo not overlap The rategy used by him is based on the 
recognition th t the c3ct ction of 1 ments in a group can be 
carried out b^ solving a set ot 1 near simultaneous equations 
using iteiati/G techniques In particular the adaptive dete- 
ction processe studied by him lai Q u e of the point Gauss- 
Seidel itecat3on n thoo Fc ha also considered several modifi- 
cations of thi bas3C detection scheme to take care of tho 
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di Crete nature of tne Qata From tho extensive siBUlation 
\A/orl carried out by him and his group [4J it app ars tl at such 
lodific tions lead to efficient and cc t effective DR schemes 
i;hose performance is comp raole to L approach Moreover such 
modifications do not require e cessivo number of sequential 
operation Extending thi approa h Clark and Ghani [53j have 
sxudied detection processe for multilevel signalling schemes 
Simulation resul-^s present d by them sho\/ that good toler nee 
o noise can be achievec oy such detection proc sees which do 
not require exces ive number of sequential opeiations 

Befoic \jincin 9 up this section we wish to take note of 
th work o Klein anc /olf [54] in which BDT is consideied from 
a different point of vi w They have investigated certain as 
p cts of h possibility of desioning a receiver niter such 
that the o/crall resoonse of the channel and the receiver fil- 
ter resembles a cyclic code generator However this line of 
thought docs not seem to have been pursuoo further 

1 3 KEY RESULTS AND OBo^rViTIOl 

As nontioned earli the basic aporoach in our study s 
to view th da a Tecoveiy problc 1 as a finite leconvolution 
proolem in the pr sence oF noise alient featur s of the 
study are briefly deacrioed in th foJioviing paragraphs 
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1 Maximum Lil'elihoocl Doconvolutxon Scheme 

tir t deiive a ML ri^cursive deronvolut on algof'i- 
thm (ML-RDA) based on Mitten s lenma [o5] wh ch pro /ides condi- 
t ons for optimal decomposition of multistage decision procG>^ses 
This algorithm performs essentially th job of ML equence 
e tination bas d on VA Hov/e/er the den /ation of the 
algorithm does not requir the use of state representation and 
explicit invocation of Bellman s principle o optimality [56] 
which are essential in deriving VA Th approach tak n provides 
b tter insight into th JlL deconvolution problem compared to the 
conventional VA aoplication We have shown that if data 
values a e assumed to be real variables the recursive algorithm 
r duces to the familiar baussian elimination algorithm for sol 
ving a set of linear equations We can th refore view ML- 
deconvolution algorithm as a gen rail zed Gaussian elimination 
algor thm for minimizing banded Toeplitz quadratic objective 
functions 

In th casG of binary data wr show that the ML-RDA reduces 
to a special cas^ of the algorithm given by Hammer et al [57] 
for minimizing pseudo Boolean fun tion We further show that 
in th binar/ case the problem of reducing Lorage and computa*~ 
onal complexity of WIL-RDA invol/cs detornination of minimal 
sum of products form of Boolean functions of seveial variables 
Therefore fast algo ithms / liable fo" such purpose can be 
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used to speed up computations and to reduce storage requirements 
Finallv we show that the recursive aigori hm deTived for mini- 
mizing pseudo Boolean functions can also b used for ML-decoaing 
of binary block codes Usefulness of the methoa is illustrated 
with the help of examples However much remains to be done in 
this direction 

2 Decision Feedbac< Deconvolution Schemes 

i/e ne'^t consider the oossihili y of incorporating dec sion 
feedbacl in DP schemes for BDT wi ti the objeciive of achieving 
good performance without going in for viL-deconvolution which 
may not be economical in some pricticai situation 

DF-DR schemes con ist of a foiAfard filter for minimizing 
the effect of future symbols a non3inear decision oevice and a 
backward filter for cli iinating the effect of past symbols 
This scheme may be v ewed as Gaussian elii ination method o± 
solving linear equations using forward elimination and backward 
substitution \ath a nonlin ar decision de/ice interposed bet- 
ween forward elimination and back substitution stages 

For deriving DF~DR chemes v;e f rst miniftizo the banded 
Toeplitz quadratic function assuming that the data values are 
real This leads to the solution of banded Toeplitz system of 
lino r equations '/e hen derive three Db-DR schemes for BDT 
The first scheme is b i ed on Cholesky decompo ition of positive 
definite matrices Tha s schem lioi/ever does not take noise 
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variancp into account The second DF DF scheme takes noise 
variance into account and i based on minimum mean square error 
criterion The bhird DF-D1 scheme is hasel on minimum property 
of Fourier expansions over finite dimensional iilbert spaces 
[58] This sneme is useful when the channel autocorrela ion 
iratrix is ilicondD tioned and vhon no kno d dge o-^ the noise 
variance is available 

Finally we derive a DF-DP schone b/ modifying Austin s 
d ection theory approach [lo] ie shovj that the modified 
Au tin s method can Iso be /leved as Gaussian ol minaiion 
method with a nonlinea deci ion device 

3 Least Squares Deconvolution Sch mes-Direct Aporoach 

'Ve then undertake a critical stud/ of least squares decon- 
volution using discrete Fourier transform (DFT) techniques The 
main feature of our study is in providing a general treatment 
to the LS-ooconvolution problem using the notion of square 
completions of the channel convolution m trix We how that 
both lS solution and the conventional DFT aporoach are particu- 
lar cases of squai completions of the ch nnel convolution 
matrix Mor'^ovor DFT approach to the deconvolution problem is 
a paT-ticular eas of k-Circulan completion o the channel con- 
volut on natri) Bas^d in k-Circulant completions we develop 
two data recov ry schemes which are easily impl mentable and 
have reduced computational conpJoxi-iy 
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Mthough DFT based deconvolution is widely used [b9]-* 

[o:)] conditions under /hich such ■'•echiiqups provide optimal 
solution to deconvolution problem accordinr to some criterion 
o not s ern to have been explicitly mentioned We investigate 
conoitions on k-Circulant completions wtich provide LS solution 
to he decon /olution problem Using th framework of squa^'e 
completion of i atrices we obtain conditions on scuare conple- 
ions that provide Moore-Penrose inveis of a full rank matrix 
Such completions are call d orthogonal omplr>tions Then it is 
shown that orthogonal I -Circular t coi letions r ortnogonal 
l-Circulant ma rices Usin this suit vj' show that for all 
practical cases of int rest Lu solu on to the deconvolution 
pioolem can not be ootained via oirect inversion of l-Circulant 
completion 

4 Least Squares Deconvolution Schene-I terative Approach 

Finally we con&i ler an iterative deconvolution scheme 
using steepest desc nt algorithm (SDA) One of the purposes 
of thi study IS to examine the ooseivation of sev ral authors 
[21] [22] [64] [66] working with fixed stop SDA in the DFT 
domain that the DFT domain iteration scheme converges faster 
than the sample domain ch me For this purpose we derive SDA 
in the DFT domain ff'om throe different viewpoints \J& show 
th t the DFT domain scheme does not alter th convergence rat 
it only helps in reduc ng computational comol xity at each 
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iteration Howevoi it i<; shorn that k-r'irculdnt completions 
can be usod to obtain good stimates of the conv^ rgen e para- 
meter in the case of some examples Th so ostinated values can 
be us d ither in the sample oomain or in th'= DFT domain to 
acc lerate convorgenc'^ 

Since the orror p rformanco of each jF schem trongly 
depones on xhe particular channel used we give an account of 
good channels for the ourpos of compa'cing the performance of 
V nous scheme Tvo criteria for jood channels are con- 
idered Th fust cri erion provide channel having maximum 
minimum oistancc (/LiD) c lied /1i iD channels Th so channels 
art u cd a be chna Is for conparmg the pt-rformance of 
various DR schemes used on ac al chann Is 

D termination of MivSD-chann Is however is a difficult 
proalom Therefor w consid r the second class of good 
channels called maximum oistanco channels ( iD chainei ) which 
rnaximi? the total aistance A procodurt is given foi obtain- 
ing i iD~channols via ^lD-channGl characterization For the pur- 
po of comparing actual channels with IMD-channols /'/g also 
a velop a lecirsiv algo ithn for computing th minimum dis- 
t ICC of a given chann 1 It is shown th L some o the AMD 
ai d \D channels art partial re ponse chann^^ls 

The pcrforianc of some of )c iata Ttcovery schomes dis- 
cussed arliG"' has been evaliatcc using cotnoj er simulation 
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Performance curves shov/ing the variation of probability of 
rxor v^th s gnal-to-noise ratio are given for a wide range of 
block 1 ngths and typical channel examples 

The sjmulition study brinjs out the following merits of 
BDT o\ er CoUT 

1) If decision feedbacl ide is used to imorove the per- 
formance of Bji systems error o'^opa^ation is limitea to almos 
1 block length in the worst c se whereas in the case of DF-Dl 
sene 1 9 for DT erroi riooagation can ex end over much longer 
duration isince in the anqe o mediun signal- to-noise ratios 
the error proo g tion is a serious proble n with DF-DR schemes 
for r^DT syston uDT with JF DR schemes are to 5*= preferred in 
such applications 

2) With an appropriate choice of the block length BDT can 
b used to provide bet+er trade-off b tween performance proce 
ssing cornolexity and data rate 

A drawback of BDT is that the effect ve data rate over the 
channel is less compared to CSDT for a given source rate The 
loss in data rate may be conpensated oy transmitting data ov i 
the channel ac a higher raie This results in incr a ed lol 
a mono the block samples Our Simula ion studi s how ver indi~ 
catc hat 1-^ h clock lungch i propc ly ciosen he degradation 
in performance o DK schem for BDT vi/ith increased ISi will not 
be Significant 
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1 4 OPGANIZATIOU OF THE TI-ESIS 

Be ides this chapter the thesis consists of 8 chapters 
Chapter 2 gives a detailed account of the assumption nade in 
the study chann 1 model coisid'^red a discussion of the BDT 
format ani problem formulation In Chapter 3 we develop a re 
c irsive algorithn for max mum likelihood deconvolution of data 
and study the application of this methodolog/ for decoding ox 
error correcting binary block codes Decision f edback DR 
scheTies are studied in Chapter ^ Application of k-Circulant 
matrices for designing cost effecbiv DR schemes is studied in 
Chapter 5 In Chapter 6 we discuss condi cions on k-Ci culants 
for providing least squares solution to the data recovery pro 
blem Application of steepest descent algorithm for LS- 
deconvolution is dis ussed in Chapter 7 In Chapter 8 we 
study the performance of some ox the DR schemes developed in 
the thesis Chapter 9 gives conclusions and suggestions for 
further research 

/ 1 o NOTATIOIIS 4 D CONVENTIONS 

In this sec ion we v/ish to mention some of the notations 
ana conventions which aro common to all the chapters of the 
thesis Pest of he noxations and conventions aie explained in 
a list included after the li t of figures Lower case letters 
with a b^T below are used to denote sample domain or time do- 
main vec ors For exanple a h x are sample Jonain vectors 
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Upper case letters with a bar below r present DFT or frequency 
domain vectors of the corresponding samole or time domain 
V ctors Upoer cas lett rs without a bar below except R and 
C rep esent matrices and I'espec ively denote iden- 

tity matrix of order m and null matrix of order mxn and 
denote rn-dimensional leal and complex vector spaces respectively 
Ue use R(*) to denote the range soa e of A - the space spanned 
by the columns of A and N{A) to denote the null space of A- 
the space of 11 v<=‘Ctors x such that *x = o Lower case letters 

without under bar are use to denote variable nd constants 

T 

Transpose of a matrix B is denoted as B If B is a complex 

matrix B denote comolex conjugate transposw of B Similarly 
x- 

B d notes complex conjugate transpose of vector B 

jj X |j derotos the Euclidean norm of the vector x given by 

I I X I I = U* 

I I A I I denotes norm of matrix A Wh-^n A is symme- 
tric I I A I 1 = max I X | where jX | is the absolute value 

l<i<n 

of the 1 h eigenvalue of n n n matrix A 

Fnd o h proof of theorems is indicated by double 
star Comoutational compJexity of algorithm is denoted 

by O(g^n)) which xs to b read as OT'der of g(n) 



18 


CHAPTER 2 

BLOCK DATA TRANS VII SSION 

In this chapter wc et the stage for the development of 
Inter chapt<=‘rs The assunptions made in th stu )y are given 

in Section 2 1 Discrete model of the chann Is used in the 

stud/ IS discu sed in Section 2 2 Block data transmission 
form t and some of its detaiJ s are given in Section 2 3 
Finally in Section 2 4 ve how that the oeconvolution problem 
IS equivalent to minim za lon of positive d finite Toeplitz 
quadratic function \ath data as unknov;ns 

2 1 ASSIMPTIONS 

Consider the data transmission system hown in Figure 2 1 
In the figure input data stream to the transmitter filter is a 
sequence of impulsesC x_ 6 ( t-iT„)3where T„ is the intersymbol 
dur tion and 6(t) denotes an unit impulse function deno- 

tes the value of the ith transmitted symbol The transmission 
path is a linear ba eband chain'll consi tinq of a lodulator 
bandpass channel and a d modulatoi tioi e is aeded at the out- 
put of the channel 

For the pmpose of stud> we make following assumptions 
regarding the data transmission system 
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1 A synchronous data transmission syston is assumed That 
IS the receiver has knowledge of the correct apling in tants 

2 The ba oband channel is linear and i is characteriz d 
b/ ar inpulsG response h(t) In lost o th prac ical cases 
channels can be modelled cither as time invariant or as lovi/ly 

imt varying In the case of tin varying channel aata 
recov ry sch mes must be made adapxive V ho evoi will not 
s udy this aspect in this the s 

3 The chann-^l impels rosojnse is of fir e duration This 
assunotion is met b/ mo t o the prac tic 1 ch nncls A1 hough 
for a bandliraited channel the impulse response is of infinite 
duration in mo t of the case ipulse response becomes n gli“ 

c bio after a finite duration 

4 Data symbols aro statistically independent and take on 
valu with equal probability from the sot {+1 +3 +5 

where q is an even integer This assumption is rarely met by 
any practical coim unication syston Howovor it can be r alizod 
t som^ extent vath the help of scramblers at the tran rntter 
end The effect of scrambling can be removed by descrambling 
the data symbols at the receiver end {see Clark [4] page 178) 

5 Uoise samples are statistically independent and each 
sample is a Gaussian random vaiiabl with zero mean and /anance 
h^/ Th white Gaussian noiso assumption is widely u^ed as it 
consioerably simplifies maohematical analysis and provides worst 
Case ocrformance 
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It ma/ bo noted the c although some of those assumptions 
nay not b valid xn pr^c ic th / provide a Js ful f amciAork 
or formulating and analysing the problem 

2 2 DISCPETE TIViE MODEL FOP ISI CHAlMNELS 

Due to the pr sence of th sampler aft r the receiving 
filter (see Figure 2 1) it is conv men! to anal/se tho base 
band data transmission s /stem vi/ith a discrete time equivalent 
node! When a finit number of da a ymbols arc transmitted 
over finite ISI channels ii he presence o vjhit G ussian 
noise it can be shown '^30] [7] that he channel lod 1 shown 
in Figure ? 2 provides an quivalent model for the estimation 
of th input sequence withou any loss o rror performance 
For this to happen transfer function of tho transversal filter 
I (z) must be chosen such that 

X( z) = H( z) ( z) 1) 

whore X(z) is the z-transfon of the autocorrelation function 
of rho channel impulse response h(t) and H(z) is a oolynomial 
hiving roots of X(z) which are in ide the unit circle 

Clark [3] has consider a chdi rcl responses doriv d in a 
d ffereni manner H as lmg that effective, multipach distor 
tion intro Juced by the transmission path can be modelled as an 
quival nt sampled impulse respense of uho linear tiansmission 
oath The ov rail simplcd r spons of th channel is obtained 
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by considering identscal tnnsfnr functions for bo h trans niter 
and receiver filters 

Fo" the purpose of study we use th° discrete time model for 
finite ISI channels sho jn in Figure 2 The lodel consists of 
a finit length transver 1 alt r \/ho e tap coefficients 
refJect the ffec o transmitter filter cha inel and receivinci 
falter Flexibility in the choi e of the tao coefficients to 
tahe care of a w d class of channels incluoing channel varia- 
tions (by macing chc lap coefficacnts vary v/ith tine) nd the 
inple manner in v</hich it captures the es en lai idea of ISI 
and noise are the major onsiderations for u ing t e model in 
the s udy 

In Figure 2 2 denotes a s quence of statistically 

independent data s/n ols denotes a sequence of independent 

Gau sian random variaoles vath mean zero and variance Ho/2 
h^ 1 = 0 1 g are the ch nnel paran t s j inoicates 

tho number of ISI components and it depends upon h aata 
rate and duration o the channel aroul e response As the data 
rate increases g also increases 

Fion the chann 1 mod 1 f Fiqur 22 it is easy to see 
tnat ch outDut sequenc has the follov/ing r presentation 

= 2 h, . w. 1=012 (22) 

1 1{=0 ^ ^ ^ 
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Data rocov r/ oroblem in h casG of CSDT systems is 
essentially that of solving the ot of oqiation giv n by 
(2 2) for s In tie ca e of long strinjs of data this 
bocomos a di ficult problem due to the largr numoor of equa- 
tions which are to be solved n h pr sence of noise Block 
data transnission xormat which o/crcoines this problo i is des- 
cribed in tho following ection 

2 3 BLOCK DATy TRWo ISSION FORMAT 

In BDT bloci s of data of 1 ngth m m h each block 
followed by g z ros is transmitted through the ch nnol If 

the source data rate s l/fg symbols p r socord due to tho g 
zero following each block th ffecti/o rate of transnis ion 
1 11 be m/(nT ) syibols per second where n = m+g On tho 
other hand no reduction in data rate is xpericnccd jf he data 
symbols are transmitted through th channel at a rate of 1/T^ 
symbols per second V'jhere ts obtained a ^ L t g 

b chosen such that the cnannel dispe sion duration (T^) coes 
not G cccd (g +1) i seconds That is 

(g +1)T^ > (2 3) 

Then i giv r by 

= n l/(m+g j (2 4; 

uos ituting tho v luu of T from (2d) n (? 3) wo obtain 


g > m (VT3)/(mT3-T^) , raT3 > T,^ 


(2 5) 
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Due to the pr^*s<=‘nce of g z ros following each transmitted 
bioci the channel is forced into zero state before the arrival 
of a fresh block at the input hus the effeci of the channel 
IS confined to blocks of lenj h n v/here n = m-^g Therefore 
ISI does not propaga o from one block to the ubaequent ones 
Dug to this f atu'^'e blocl s can be processed ind pend nti/ at 
the receiver to recover data /alue Note that two buffers 
will be needed of which one wiJ 1 0 ° receiving the next block 
hile the oxher holds th curient block for oroce sing 

Figure 2 3 helps to nake some of the ic s expressed in 
the previou paraaiaphs more clear In Figu 2 3a a rather 
simplistic channel is shovn for the purpose of illustr-ation 
Th Qata duration is assumed to be one third of tne duration 
of the impulse respons Oucout of the channel shown in 
Figure ? 3a to an arbitrarily chosen s+ring of binary (±1) 
data is shown in Figure 2 3ta Figure 2 3c shows the corres-" 
ponding o/erlapoing pulse responses Figures 2 3d and 2 3e 
show tho corresponding situation in the case 01 DDT wixh 
increased da a rat over the channel In this case m = 6 
g = and n = 10 Moreo/er w obtain T. = 0 6 T_ as the 

data transmission rate ov r the channel From the Fijuros 
/ d and 2 3e it c n be seen tha the blocl s get separatee at 
th channel outpu-1 

Cer am features of the inequality (2 5) can be made 
clear with a graphical repr entat on For this purpose 
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let p = Then (2 o) becomes 

g > (p-l) m/Cm-p) m > p (2 6) 

In Figure 2 4 g i plo ted against m. lor variojs valu^^s of p 
v;h n g is given by 

g = (p 1) m/Cm-p'' m > p 

N tc that althouQh n an g a'^o in teg rs in Figure 2 4 

in order to avoid oo many >mbols g is plotted as a continuou 

function of m for m > p From the figur it s clear tiat for a 

g3 ven p m must be sufficionLly large for g to be small Under 

this condition will b close to T„ /e al o observe that 

c s 

the requiic value of g lev Is off fast after dropping rapidly 
n th beginning This m ans that thr> effect of I3I among the 
memb rs of a block will not become too large if moderate value 
of n IS chosen 

2 4 THE DECO IVOLUTION PPOLLLM 

In this section we formulate the d convolution problem 
L t h be an n~l ngth vector vuth first g elements as the 
channel par mot rs and ros of bn olcnet ts zero Let x denote 
an in-lcngtl oata vector and ran length obs rvatDon vector In 
terms of th se v^ectors h convolution relationship given by 
(2 2) can be /ritten as 


r H X + 


(2 7) 
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FIG 2 4 


Plot of d' v«''sus m 
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v/h P 1 th con /olu ion ma t i ;ith h as ti •fir t column 
For th sai'c o illustration H matu is ho\;n b low 



rl “ 



h. 


0 

0 


h 

0 


9 





0 


n. 


he (i j)th el II nt oj: H is given by 




'■ ” |i~a| ^ ^ J 

0 for i >3 an I i-j > g 


1=12 3 


n 


1=12 3 


m 


The oroblom of d 1 irnining x given r and h or finding h 
giv n r and x is call d finite d convolution orobl m in tho 
pi sence of noise 



30 


1 o Droble~i has received as much attention as tho finite 
ieconvolution problem due o its ppearance in a wide /ariety 
of applications [59]-[61] [67]-[71] However most o the 
methods available for dealing with the problem a ume that the 
uni nown ya'^iables are reaJ Vilhereas in the context of BDT 

th dat vector x cake di cr te values fron a inite alpiabet 
Therefore in this case / have to deal v; bh a coirbinatorial 
ae onvolution problem in which the pace oi all possible solu- 
tions do not po sess the nice proo riies of r'^ to \jhich real 
vectors oelong One straightforward method of deconvolution 
IS t search through all possible solution However even in 
the binary case searching through all possible 2’^ vectors 
becomes out of hand for modeTate values of m ivioreover due to 
the pr s nc^ of noise no xact solution is possible in general 
The usual aop oach tak n under such situations is to chooso a 
vector X which optimize sone suitable performance criterion 
Depending on the constraints put on the solution space one 
obtains different s lutions to the deconvolution problem in 
the presence of noise 

In d ta CO inuni cations the nos souglt after criterion is 
the ninimixdtion of tl rroi probability For he case of 
qually lik ly data vector it can be shown that the aaximum 
likelihood (tlL) segue ice osfcimation provides mini jum probability 
of rror [7] fho ML ostiaate of x i that vector which ma/i- 
mi es thr conditional density d(i/x) 
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Old r the assumptions mad in oection ? 1 th conditional 
dens ty function of r niven a particular data vector x can be 
hovm to be 


o(r/x) = ~ exp( ^ (r 

1=1 ^o ^ 


g 

=0 





it IS Gas/ to see hat the abov density function is maximized 
by minimizing th qiadratic function 


F(x) = (r“H<)^ (r Hx) 


m 

= £ 
1-1 


n 

2 

3=1 


( X d I I 

13 |i 31 


m 

1=1 




m 


h 

1-1 


(2 8 ) 


where a | is cho (i 3)th elon nt of and b^ is the ith 

T 

element of tte vector 2H r 

I'hereforG the docon /olution problem in our case is equiva- 
lent to mininization of th quadratic function given by (2 8) 

sub3 ct to the condi ion that the unknown variables take 

values from a finite alphab t oince H is a convolution matrix 
it is easy to s e tl at H H i a symm trie Toeplitz matrix \ife 
v;ish to note that a squaie matrix D is called Toeplitz if 

^13 ~ '^U-3 I *^13 olonent In addition as 

T 

g IS less tht,n n H H is also a band Jl i dtxix Thus the decon- 
volution problem is e sentially that of minimizing a banded 
Toeplitz quadratic unction sub3cc to different constraints on 
the unknown variables depending on various performance criteria 
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CHAPTER 3 

MAXlViU?i LIKt^IHOCD D CONVOLUTIaJ 


In this chapter we use Aitten s lemma [55] to derive a 
maximum likelihood recursive deconvolution algorithm (ML-RDA) 
v^/hich essentially performs the job of Viterbi algorithm (VA) 
[38] Ml ten s lemma may be viewed as a more iormal statement 
of Bellnan s princivil of optimality [56] and it clearly stipu- 
lates the conditions for optimal decomoosi tion of multistage 
op Dmization problems For the sake of easy reference Mitten s 
leifuia IS given in Appendix A 

The chapter is organized in th oliowing manner In 
Section 3 1 we use 4it en lemna to develop a general recur- 
sive algorithm for minimizing decomposable nonlinear function 
of several variables A specific case when tro nonlinear fun 
ction is a quadr tic function s taken up in Section 311 It 
1 shown that minimization of a quaoratic function of real 
V enables using the r cursive algorithm is equivalent to 
Gaussian elimination algorithm for solvina a set of linear 
simultaneous quations The discret version of the algorithm 
1 discussed in ooction 3 J 2 The ML-RDA for a banded 
Tooplitz quadratic unction (BTQF) is given in Section 3 2 
For the puTooso of comp i on in See on 3 3 wq discuss VA 
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for minimizing BTQF Strategies for alleviating storage and 
comoutational requirements are discussed in Section 3 4 Appli 
cation of th se strategies to Binary caso is considered in de- 
tail in Section 3-^1 Application of the recursive algorithm 
for decoding of block error correcting code is considered in 
Section 3 5 

3 1 RCCURSIVh ALGORITHiVl FOi llIlMI IIZATION OF NONLINEAR FUNCTION 

OF SEVERAL V PIA3LES 

In order to illustrate th principl of ML RDA we first 
give a method for minimizing a nonlinear function of n varia- 
bles Let ^2 ^n^ ^ function to be minimized We 

assum that th function is dcconoosable in the manner indica- 
ted below 


F^(x) = F(xj^ X2 
^3 


xj = U^(Xi Xg 


n 




x„) + 


(3 1) 


Note that is func ion of x^ x^ alone Now 

suppose that inst ad of mininizing F(x) with respect to x 

ve can minimize Uj^ with respect to x^^ first and then minimize 

the combined function min X 2 x^) + Vjl(x 2 Xg 

^1 

with r spoct to X 2 Xg Xj^ Put differently we wish to 

compute P 2 where 



[V, (x 






n 


Xj^) + min Uj^Cx^ X 2 

(3 2) 



3 ^ 


At this stage a valid question to ask is whether or not 

P- = min ^2 ansvjer is y s then 

>^1 

1 n 

eff ctively we have decomposed the n variable minimization 
problem into a single variable minimization problem followed by 
an n“l variable minimization problem Thus it is essential to 
know whether we can distrioute inn operator as indicated in 
(3 2) vatlout affecting the op mum solution 

In Lemma 3 1 we show that whenever F^^^l is 

decomposable in the manner giv«n by (3 1) 


Pg = min F X 

>^1 Xn 



vhich efrectively proves that the min operator can be distribu- 
ted in the manner shown in (3 2) We v^^lsh to note that 
Lemma o 1 is a sp cial case of the more general decomposition 
result giv n b/ Nitt n [55] for multivariable optirniz tion 
problems A proo-^ o Mitt ns result tor two variables may be 
found in [56] 

Lcmna 3 1 Given (3 1) and ( 2) Pg = min ^l^^l ^2 ^n^ 

^1 

Ptoo-f^ Let P, = nin F-,(x-, x ) 

x, X ^ 

1 n 

From the definition of t nirium value we have 


P 


2 


> F 


1 


(3 3) 
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Uhonev r < Uj^(x^ x^) vifi 

Xo X fix d 1 folloAfs from (3 1 ) (sine 

O f i 

V^(/2 ^3 fixed) that 

Bit fo each value of X2 x^ x^^ 

Uj^Cxj^ X2 Xj^) = min U^(xj^ x^ x^^) < U^(xj^ >2 

who x^ IS th valu of at whi'^h X2 x^^) 

attains minimum value \;non X2 x^^ are fixed 

Ther fore from (o 4 ) w c t 
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From ( 3) we thus get = P, 

Let Xj^ be thi=> value of Xj^ ;hich iiinimzes 
a fi/ed X2 No that Xj^ b comos a function of 

Xj X3 Let 

=‘i= 9 i(^ 2X3 

Substituting for Xj^ in Fj^( X2 x^) the minimiza- 

tion probl m now reduces to a n-1 v riable minimization problem 


rain 

^2 


F2(x 2 X3 Xj^) vhere F2 (k2 X3 


Fi(Xi X2 


= 

^n> 


Assuming again that F2 is decomposable we can write r2 as 


^2 ^ ^2 ^3 


X„; = U2(X2 X 3 


Xt,) - V2(X3 X4 


’'n) 


Using Lemma o 1 we can write 


min Fo(yo Xo 

X2 Xn 2 2 3 n 


[V2(x3 


''r>> + 


n 


min 1^2^ ^2 '^3 
^2 




Let X2 be the value of X2 which minimizes d2(x2 Then 


X, = g (x _ X, 


='n) 


The r cursive proc dure no v can b easily written for the ith 
stage as 
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f'a'-'i ^+1 




* 




min ^1 “ 

^1 ^n ^1+1 ^n 

^'9l(^+l 




\ ) 


X ) + mn U (x 

n' y 1^ 1 

1 


X )] 


X = 1 2 


n 


At the nth stage we obtain 

min Fj^(>^)=nin Uj^(x„) 

X X 

n n 

Th refore x^^ is a constant i e is a constant function 

oubstituting x^ in can be obtained Continuing 

this procedure of back subs titu cion all the x^^ i = 1 2 n 
die obtained 

It IS nob dif icult to see that the foregoing approach is 
\f ry similar in spirit to the Gaussian eljnination method of 
so3vinq linear inultaneojs equations which involves forward 
lamination and back substitution The collection of functions 

Xj^ ~ 9t ( ^2 x^) 

^2 ^4 ^n^ 


= 9i(x 


1+1 ^ 1+2 



Xn = g^^ constant 
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IS rernxriscent of th upper triangular m tri v;hich results 
after forward elimination In viev/ of the foregoing observa- 
tion we wish to call the recursiv algorithm oev loped above a 
g neralized ("aussiar elimination orocedure 

In the proceeding algo ithm ve implicitly assumed that the 

indic ted single variable ninimization is Iways possible That 

IS mn U (x a) o ist for all i and f xed x , x 

y i' 1 n' 1+1 n 

1 = 1^2 n ho e /er this may not alway h oossible for 

an arbitrarily specifi d nonlinea tur tion in afdition sin- 
gle variable iiiimi ation itself may be difficult to carry out 
K v rtheless this approacn has several advantages in tho case 
of the ML”dcconvolution t; roblom ormulatcd in Sec 2 4 Before 
discussing that case we wisn to cons d r the probl m of mini- 
mizing a quadratic func ion of n rea variables as an illustra- 
lon of the orogoing recursive algorithn 

3 11 Real Variable Case 


For the sake of simplicity we consiocr n = 3 
tion to n variables is straightforward Let 


Generaliza- 


3 3 3 3 2 

F,(x-, Xo) = -2 i 2 X X a + 2 rf 

1=1 ^ ^ i=l 3=1 ^ 1=1 ^ 


and ” ®3i for all i 3 

where b^ a^ and r^ 1 = 1 2 3 are constrants Carrying out 
the decomposition of ^ with respect to Xj^ w obtain 
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Fi = X2 x ^) + ^3^ 


/.here x^ X3) = + 2Xj_ -h xj 

Xji^ whi h minimizes Uj^ /.i h x^ x^ fi/ed is given by 


’‘1 - Xja ;^ j)/ail 


assume that aj_j_ 5^ 0 Substituting in Fj^ we obtain 


3 3 3 3 

F,(x, X,) = -2 E X b + E S X X a + E + (r, 

1=2 ^ 1=2 j=2 ^ J J 1=2 ^ '■ 


wb re = b^-b^ ^ii/aii ^ ^ 


( ri )2 = r ^ bf / a ,. 


ano 3 , ~ 9 , Si. 

ij ij 13 13 1' 


1 j = 9 3 


low r2 ( X2 X3 ) = U^(Xo X ^) + VryiA ^) 


2 \ a2 ^ 3 . 




v/h re 133(^2 ^3^ “ ■*■ ^^2^3^23 ^2^22 

therefore X2 = ^'^2“^3®23 )/ ^2^ ^22 ^ ^ 

u mg X2 we obtain F3(x2) = “2x3b3 + X3a33 + (^1^) + ^^2^ ^ 


ninimization of F3 gives 

X3 = b3 / a33 83 ^ 0 

■55- ^ ^ 

Back ubstitut3on of 3 givos X2 n ch in u n gives 


CM CO 



The foregoing example clearly shows xhat the recur ive 
algorithm for minimization of a quadratic function or n varia-" 
bles IS equivalent to Gaussian elimination technique fox solving 
a set of linear equations 

312 Discrete Variable Ca e 

Haviig di cu ed certain aspects of the multivariable 
minimization with real variables wo nov; conside the case when 
he /ar ables take values fcon a inite et Without any loss 
of generality we consider tne quaoratic func ion used in the 
p evious section with the d fcrenc that th variaules take 
valjes from the set S = {-3 -2-10123) Cxtcnsior to n- 
vanable case and any given firite alph u t is straightforward 

Following the steps as in the reaJ variable cas in tho 

first stage we have to -^ind min ^2 '^ 3 ^ this case 

^1 

differentiation i not possible Hence wc must tabulate 

values of which minim ze for fixed and Xg The 

tabulation format is shown in Table 31 No of en ries in th 

2 2 

first column of th'=» table will be 7 corresponding to 7 possi- 
ble values of and x^ Moreover for each set of X 2 and x^ 

; iu t ch ck 7 V lu s of Xj o decide x^ Thus th first stage 

3 

its^^lf in/ol/ s 7 evalu tions of Furth r in general it 

«• 

IS not possible to expr'^s ^ closed form as in the caso of 

1 1 variables Therefore can not b*^ uostitutod in Fj^ to 

(• 

00 am This implic^ that \ie must use h t ble for in 

th second stage In th^ socona stage we write F 2 as 
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Tnbl 3 1 Stage 1 Tabl 3 Stage 2 





-3 


3 -2 


■2 


3 -1 


-1 


0 


3 


1 

2 

3 


For each value of ^2 and X 2 vi can uso from the table 
for Thu in tnc so ond stag<^ we minim ze U 2 +U 2 (X 2 Xg) with 

r spcct to X 2 by fixing Xg Th re aio 7 different values for 

fr 

x^ rd foT oach Xg X 2 is to bo decided on the basis of 7 possi- 
ble valu s of y 2 Thu the second table will *'ive 7 ntries as 
shown in Taol 3 2 In a sintJar fasiion we can carry out the 
computation required ir stage 3 with th*- help of Table 3 2 of 
stage 2 

Lot us no V xamino the comoutational conploxity for n- 
variable as^ w th ach variable taking q discrete values In 
the first tage ev luations of is required Likevi/ise in 
the ixh stage q evaluation of required On the 

oth r hand a dir c t search method v/ould require q^ evaluation of 
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th funct on F It thus becomes clear that for the scheme des- 
cribed to become conputa lonally efficient the unction al forms 
of U2 must be much inpler than the functional form 

of F In otherword U2 must have the followiiig 

desiT'able oroperties 

1 ® fiust be functions of a lail number ox /ariables 
compared to the total number of variables involv d in the 
minimization problem 

2 ® must be easily computabl By this wo mean the number 
of addition and multiplications needea for evaluating the 
function nust bo s lall 

/^hen bo+h the properiie are sat sfied by an objective fun- 
c ion it 1 possible to reduce the number 01 computations and 
storage requirements substantially using the foregoing recur- 
sive algorithm compared to the direct enumeration approach It 
may how ver be noted that storage requirement is still large 
compared to the enumeration approach 

The obj ctive function in the cas of ML-deconvolution 
probl m formulat d in Section 2 4 is a special type of quadra- 
tic funcxion called banded Toeplixz quadratic function Such 
a quad itic function me ts th above mentioned requirements In 
the next sociion we derive a JiL-RDa or this case 



43 


3 2 RECURSIVl ALGOiilTHi FOR BANDbD rOFPLIiZ QUAD ATIo FUNCTIUNS 
^fe recall from Section 2 4 that the function 


m 


F{x^ 


m m 


x„) = 2 2 X, b -h S S x,x,a I, , i-^-c 

1=1 1=1 j=i ^ J I 


where a | = 0 U~3 1 > <^ g < m 


IS called banuej To olitz qu oratic unctioi (3TQF) for the 
marrix formed by a |i„j | as eJements is a band d Toeplit 
matr x 


I et F 3 _(x) == r(x) r the sake of uniform ty 
decomposed as 


can be 


■1*^1 


+ [-2 


g 1 


+ 2x1 

^1® ll- 1 

1=2 

J. jX**' X 

m 

m m 

- b + 

Z S 

=0 1 1 

1=2 1=2 


. 2 
^1^0 


= X2 X3 Xj 

Ilote tha Uj^ is only a function of g+1 variables Further 
IS a function of consecutive g+1 var ables rather than g+1 
arbitrar variables as it may happen in the case of a general 
obj ctive function This is a specia ^jroperxy of BTQF In 
addition the form of U ^ at every stage is sai e Thus the same 
type of comoutation proceoure is used in evei> stage Through 
the folJoving example we have tried to illastrat various 
aspects of the algorithm The eyamole is from ['’1] 
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xa aole 3 1 


7 7 7 


Let F 


!I 

X 

-2Syb+i Lxxai 
i=i 1 1 i_i 3=1 1 3 u- 

'3 1 

whe^e 

'^1 

= 1 a ^5 ~ 

1 5 

ciq= 1 aj|^=0a 5 ^ — “0 

25 



= 20 b^ = 

-3 0 

g = 2 



^3 

= 05 b^ = 

0 b 




^4 

= 10 




and 

\ 

1=12 

7 

t ke +1 or -1 values 


tage 

1 

^l^^l '^2 

x^; 

— ^2 ^3^ ^1^^2 ^3 

X7 



^l^'^l ^2 ^3^ 

= -2x 

^ 2 

vii-xr -<1% 



1 2 

= -3\j^ + 2/j^[0 5 2”0 25 X 3 J 


T ble 3 3 IS obL inetl by uetornining th value 01 Xj^ that 
minimizes for a fixed X 2 "^nd Xj 

Table 3 3 Stage 1 

__ «■ ->t“ 

^2 '^1 

-11 1 -5/2 

-1 1 1-3 5 

1 1 1-0 5 

1 1 3-15 
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Since 

stage 

Stan 


= 1 for all th values of ^3 can decide at this 
itself that the fust vaiiabl is - 1 


2 '-J2(x2 X4) = “2x2^2 + 2x 


4 

2 ,^3 






As the calculatiois arc dentical in all the stages we 
have presented th f nal results in Tabl s o 4-3 8 correspond- 
ing to stages 2 to 6 


Table 3 4 Stage 2 


X3 X4 X2 




u 


2 


“* 1 J. 1 »« 4 


11 1-5 

1-1 1-3 


11 1-4 


Therefore the second variable 

X 2 = 1 


Table 3 5 Stage 3 

X4 Ug 

-1 -1 1 -3 5 

-11 1 -4 5 

1-1 1 -45 

1 1 —1 — ’sj o 


Table 3 5 Stag 4 



-1 -1 


1 -6 


-111 


-7 


1 -1 


1 -3 


111 


-4 
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Ta Ic 3 7 Sta-je 5 Table 3 0 Stage 6 





-1 -1 


1 

-7 


1 

-1 

9 5 

-1 1 


-1 

-6 

5 

1 

1 

-12 5 

1 "1 


-1 

10 

J 

1 

-1 

“2 0 

1 1 


1 

9 

0 

1 

1 

-1 5 

From 

Taol 

3 o 

AQ 

th 


= 1 ana 

= 1 On back 

ubstitution 

Ta Die 

1 

giv s 

X = 

0 

1 It may Is b not e hat 

Table 3 ^ 

no 

labl 

3 7 

or stage 4 

and 5 r 

soectively diioctly 

inoica+e 

ha t 

II 

1 and 

y = 

-1 

fsing tho 

values of and 

in Table 3 

we 

obta 

n X3 = 

= ~1 




u o VITbRBI LbOPITH.1 TOR iINI lIZIi G QF 

Foi th'^ purpos of co oar son we hav given a bii f des 
eruption of VA in this section 

R call that the ^TQF nin Tiization p obJ cm arose in 
conn ction with Iran is ion oi digit 1 can through a finite 
imi ulse respons han iv, modelled as finite oino y transversal 
•^iltcr 

At tho kti nnstart he outaut of su h ilt r can bo 
repr sented in terms pu Xj^ ^k~l '^k-g 
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q 

E 

1=0 


\^k- 


k - 1 ? 3 


Vi/h r g IS +h mcnory o \ transvrr dl filter Thus y^, i a 
functioi of the pi sent input as v/ell a the previous state of 
h transversal filter Let denote tho tatc of the trans- 
versal flit r at th t th ii tan Th i 

\ >=1-1 Xk >=k = ° If k < 0 

In t rms of s yj can be expressed a 


Yk = f(ho '>1 




^k-1^ 


Th 1C foie given nn inpu sequence {x,^} wo have a correspond- 
ing st t sequence and an output soqucnce £ 

CO respondtnee is one to one This observation d crucial 
to xho dovelopmnnt of VA Due to this corr spondonco we can 
niniinize F()<) ov r s instead of Xj, s For this pumose 
decompo c F(x) as 


yj^} This 


n 


ni 


F(x) = f^(x) = [-2 

[-2x^b^+2X3^(x2aj,-/3a2i- -r x^a^] 


Note that the quanrity in ide tho second nai r of biackots is 


only 1 function of Xj^ X 2 x„ •, Le 


fl(k) = Uj(Xi 


Xg^i) + Vj^(X2 


^m> 



^8 


It IS easy to see that ^9 "^g+l^ ~ 

Nov Vj^ can be si ilarly decomposed a 

^3 ^m^ ^2 ^^3 ^4 ■*■ ^2^^g+2 ^g+1^ 

In a similar fashion we can v/r3 te for the kth stage 

''k(>=k+l = 'l<+l(>'k+l ’'tr.) '^k^Vl' 

k = 1 2 3 mg 

Therefore r(x) can be writ en as 

F(Si Sg = 2® g(S^) 

wher 

g+k 2 

'^1 ^ g-rk “^g+k-l^ ^^k j k-j | ’^^k^o 

k = 1 2 3 m-g 

We wish to note that the derivation of the foregoing recur- 
sive equations is basea on the aporoach given by Hayes [41] 
The optimality of th algor thm lollows ■^rom Lemma 3 1 

The algorithm works as fol]o/s In the first stage of 
the algorithm S^) is maniniz d with respect to for 

fix d ^ ^g+1 assume anyone of the states we 

have to s ore value corresponcinj to sta es As 
and differ in only one pi ce d ternination of optimum 

giv n IS off-^ctively one Vdriab3e optimization To 

complete our discussion let 
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r(s 


g 


SJ = S,) = 5g) + 


Hence 




n-a 


nun ^ = nun Sg^^„j^) + v„.g(S^) + 


■^g ^m ^g+1 


m 


g 


"i" “i^Vi 


g 

Define _j^) u* = 0 


Th n th recur i /e e uations for /A an b vritten as 


4 = “" Vic-l' “k4 ''~12 m-g 

g-rxC*" 1 

Finall/ the ootimum Vdlue of F F i g ven as 

F = min [u^ ^ ^ (S„)] 

m-g mg m ■' 

“^m 

Result obtained by apolying VA to E a nple 3 1 is snown in 
he F-gure 3 1 In t e ficure w'=* have shown the survivor paths 
hrough the trelli In h figure dotted lines indicate alter- 
nate survivors giving same function values From the figure it 
nay be noted that (11) i node common to all the survived 
paths in stage 3 Th s iiplies that ~ ^ Thus decisions 
regarding data can be maoc \^en before completing the forward 
"ecursion process This is due to the merging of the urvxvor 
o ths through the Trellis 




Let us now consider the solution obtained by applying 
IlL-RDA to Example 3 1 From the Tables 3 3-3 8 w can easily 
see that = 1 Sirailarly table given for stage 2 gives 
X 2 = 1 The third stage shows that =-^4 Fourth stage 
gives X 4 = 1 and fiftn s cage = -1 These aspect do not 
become clear from the Trellis diagram s lown in Figure 3 1 

3 4 TRATECICS FOR REDUCING COMPUTATiO AL AND STORAGE 

RCQUIREi^ENTS 

Throughout the discussion of th recursiv deconvolution 
algorithm we have used tables to how the results obtained at 
each stage of the algorithm In actual practice su h tables 
are not necessary Notice that the tai le used in the first 
stage is used only by the second stage Therefore in, the 
second stage //e can use the updated v rsion of the first table 
For the purpose of illus cratior w have shown the development of 
the tables for Example 31 in Figure 3 2 Back substitution 
gives the path snowing dotted lines in ho figure It is obtain 
ed as follows and ar decided by noting that is 

/ -H- . . . 

min mum at that point x^ for the state (x^^ Xy) = ( 1 1 ) is 
-1 X 4 for the state (x^ <^) = (-1 1 ) is 1 "t^he state 

(X 4 x^) = (1 -I) IS -1 X 2 and are obta ned similarly 

As the pioceduro is same for a general n-variable BTQF we 
see that the storage requirom nt is registers of length (m+l) 
arh for storing s at each stage and the accumulated minimum 



Figure 3 2 Developtn nt of the 


hf in 
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Stage 2 


^1 

^2 


^4 

^2 

1 

1 

-1 

-1 

-4 

1 

1 

-1 

1 

-5 

1 

1 

1 

-1 

~3 

1 

1 

1 

1 

-4 



1 -1 -35 

-1 1 -45 

1 -1 -45 

1 1 -3 5 


^6 Stste 

1 -1 -9 5 -1 -1 

l"” ^ ^ 1 -12 5 -11 

1 “ -1 -2 j 1-1 

1 1-15 11 



53 


Vd^ue at every stag Thus the storage requirement jro /s 
linearly viLh the bio k length Further since at every stage 
e need function evaluations the computational comolexity 
of the Igorixhm gro ;s linearl/ with n'ock length ard exponen- 
tially vjith ch nnel me ory 

Even with the foreaoing strategy for storing the tables 
t e alg rithm still re iires a large amount of storage ana 
computations foi even mod rate block lengths Thus if th 
recursive algorithm is to b come acceptable it must hav 
additional features fo-^ minimising storage and computationa] 
require nents Towards this end w now propose and oxamino a 
method for reducing tho complexity of Ml-RDh which is evoected 
to ro ult in a greater savings in storag and computations 

To b gin wixh let us examine Taoles 3 3-3 8 -^or Examol 3 1 
For instance consider Table 3 5 obtained at the 3rd stago 
Note thax if v/e are able to /press Xg as a function of and 

•it’ 

x^ substi uting for an tho objective function eliminates 
the need for the table in th-^ n xt stage comoutations Moreover 
for tho purpo o of b ck substitu ion we need retain only the 
functional form of x^ In this case it is oas/ to sec that 

•it 

= "•X4 

If at every staoe we aro ablo to obtain such simple 
r present tion stoi go rtquir ments are reduced to a minimum 
In the example under consideration all the func cions have sinqale 
representations as given below 
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tag 3 2 3^5 6 

Funcxions y^~l x^=“l x^= 1 and x^=L 

c>u h 1 pic functional for is cannot always be guaranteed 
For tho L r G l eing s unie that such roprcsontations ar possible 
Then tlo next qucsti n v/ould be cm we arrive at the rurct onal 
forn Vviithout nunerating all possible, values of the fi x d 
van 1 s? If tais i possible n woulc a3so h v achieved 
reduction in the computational co olexit/ 

However in g n ral this ppoars to oe a difficult pro- 
bio In the fir t ploc funccic. i i form of may be coraoli- 
cat d duo to which storag f uch func on as vi^ll a substi- 

ti ion irt tho objoctiv functioa may b difficult Seconoly 
1 + may bo difficult to directly obt r the functional forms 
Therefore to be^in with Diastead of considering the most 
general ca c w study th binary case in ch icxt soc un 

4 1 Binary D \ta Transmission 

A?h n the data symbols assume on of th two values 1 or -1 
the objective unction can bo converted into another function 
called ps udo clean function (PBF) [57] which is a function 
of Boolean variahlos vi/irh re 1 coe ficients For this we us 
tho tr nsfornati n y = (l+x)/2 (which mo ns y takes 0 nd 1 as 
X trikos “1 and 1 resp cti/oiy) With the help of this t'^ans- 
ornation the qu dratic function F(x) can b'^ coavort d into a 
PBF F(j;;) wher y^^ y 2 are Boolean variables 


Le F(y) expressed in the matrix vector form as 


F(x) = “2x^b-^<^ 


;e use th rransf ormation ^ = (l+x)/2 


where 1 is a vectoi of all ones Vith this t an foliation we 
obtain 

r(:z) = + p 

Inhere b = 2(b+A£) A = 4A p = pi^b+l^ <\l+r^x 


The con tant term o can be i r lored fo’' he ourpose of 
minimizing F(j||') 

r(;£) xs sale as r(><) Therefoi the \L-PD<\ given in 
3 ction 3 2 applies to this case with the di'^ference that 
the tables at each tage now contains Boolean variables As 
ou-^ interest is in xpressing in terms of ^i+2 ^i+g 

at the ith stage we observ that the table in this case is a 


truth table for in terms of ^i+2 


^i+g using 


re ults from the theo y of Boolean functions [72] vje can repre- 
sent in sone convonienx form For ms ance minimal sum of 
produ t or product o sums form 


Ttus «e have shovin that in the cas® of oinary data there 
e<ists a procedure for expre sirg x^ in t ms of ^i+2 

^i+g stage Howe/p^r simple expressions for may 

not always be obtiineo Let us consider tho worst case where x^ 
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IS rcor sented in terms of ^ The truth tabl 

or will contain 2^ ntries In this case h stor ge can 
be roducod to some extent by obta_ning a mi imal represents 
tion of x^ [ tlods such as Quin'^-r\cClausky oi any such 
similar m thod [72j could oe used for this puiposo However 
jc may lave to use a special purpo c compute^ vhich imolement 
Boolean function mininization algor thm 

Now VC cons dor rhe probl m of obtaining unctional ropre- 
senta ion for withou onun ating all po ible 2 cases xiy 
using an algorithm d vc loped by Hammer et al [57] we show that 
the computational conplnx ty can be reduced it very stage 
vithout sacrifying optimality Towards this end we use 
L nun 3 1 to derive a r cursive algorithm developed originally 
b/ Hammer et al [o7] for minimizang nonlinea PBF s in the 
next section Tho approach taken here i much sinoler compared 
to tho derivation given in [o7] 

3 4 2 Minimization of Pseuio-Boole n Functions 


Consider a PBF ^2 variables Since 
psoudo-Booloan function is linear in each of its variables wo 
can write 






V 1 ( Xrj X/ 


m' 
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Application of Lemna 3 1 qiv°s 


min Fi 

X, X 

1 m 


Now since 



min Xj^ ^3 ^m^^ 

^1 

s are Boolean variabl-^s it follows that 


If 


^3 

^m 

if 

Ul(x2 

^3 

m 

if 

Ui(x2 

X 

0 

Xr 


where 0 is a don t care x/ariablo (Xj^ arbitrary) 


Therefor a Boolean function representation ror x^ can be 
obtainso by determining those y^2 ^3 which < O 

oubstitution of into gives The other stages of the 

rainamization follow the sane pattern as in the case of recur- 
sive procedure described earlier Thus at the ith stage we 
hive 


min 


X 


\+2 


■ ’‘1+1 ’‘m 




min X U (x X 2 




and 


* 


0 if U > 0 
1 

01 = 0 


1 IX < 0 
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we are interested in only one solution 0 can be fixec. 
as 0 or 1 and we can solve ^ O or obtaininj or ^ 0 
xor obtaining 5?^ Hanimer et al [57] have also develooed an 
algoritnn or determining all solutions whicn satisfy ^ 0 
This algorithm can be used to reduce the computational comple- 
t / a each stage 

From the foregoing discussion it is clear that in the 
binar/ case it is possible to r duce storage and computational 
comolexit/ of the RDA if fast processors are available for 
minimization of Boolean functions and for implementing Hammer 
t al Procedure for obt ining 

It may be noted that deter nination f iini lal representa- 
ion of Boolean functions is a c^as ical proolem in the theory 
of switching functions Although a large number of methods are 
available [72] no dofinito claim can be nad in favour of one 
aporoach over th'^ other 

In summary the over 11 effici ncy of the if impl - 

m ntod in tho manner described in this ection depends to a 
great extent on the availability of fast algorithms for 
obtaining minim 1 ropres rtations for x^ either in the Boolean 
func ion form or in the pseudo-BooJ an function form 

In the nonbinary case however no sech claims can be made 
This aspect needs furrh r investigation 
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3 5 P^CURSIVE ALGOi ITHM FOR DPCODING BLOCK ERROR COPRECTING 

CODES 

As an application of the recursive algorithm developed in 
the previous sec cion we consider maximum likelihood decoding 
of binary block codes Our approach consists in formulating 
the iv\L decoding prol lem as an equivil nt pxobler of minimizing 
d pseuao-Boolean func ion of severa"* variables It may be noted 
that the algorithm oe / loped in this section can also foe used for 
maximum likelihood uccooing of convolution cod s 

We wish to note hat Volf [73] has studied the applica ion 
of V/ to decode blocl cooes by con tructing Trellis dD agraras 
The loproach consid r d here doe not require Trellis constru- 
ction and it seems to provide a wore guneral method for maximum 
likelihood decoding of binary block codes 

The codes considor^'d here are over bF{2) where GF^2) 
denotes Galois field with two elements 0 and 1 [72] For such 
codes th^^' ML decoding is quivalent to solving th set of linear 
equations (^x = r over GF(2) such that the Hamming distance 
between Gx and r is mininum when all the code words are assumed 
to equiprobablo The matrix G is called the generator matrix 
of the code in the parlance of coding theory [6] [7] The 
received vector r is an additive combination of the transmitted 
codevector which is not observable at the receiver and the 
noise vector w That is r = £ w where denotes addition 
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operation in GF(2) It may be noted that thu lements ot i 
and w are e ther 0 or 1 We also assume that c is an nxm 
m trix 

The Hamming distance between two vectors x and ^ over 
GF(2) is defined as the number of nonzero entii s in x 
It is also known as the weight of 0 yd noted as wix © y) 

Since GF(2) elements are also elern-^nts the real field 
we can show that 

where p ^ denote pointwise multiplication Afe now make use of 
the property that if y s 0 1 real vector then 

w(x) = iJl 

whore 1 ps column vector of all ones Therefore it is easy 
to see that 

w(£ ^ p[) = w(p)+w{q) 2w(p £) 

Thus maximum likelihood decoding oroolem can be formulated os 

•nin(w(Gx)4w(r)-“2w(Gx r)) 

X ” “ 

= mi n ( 1 ^ ( G^ r ) ) 


(3 5) 
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vioxo that Gx IS to be computed ov r GF{2) Therefore 
we n ed a suitable representation for th elements of G_x Let 
82 denote n rovis of G Then we can write Gx as 

(aj.x ajX a^x) Wh re a^x = ffi 

iJ c n US the r lation 


u 0 V = u+ / - 2uv 


(3 6 ) 


in ord r to con /ert a^x into n equival nt ps udo~Soolean 
function However uch a representation boconos too comoli 
catod Afh n moTc variables rc involved A more comoact form 
of representation is n c ss to obt 11 a ri-prcsontation which 
reflects the gon rator matrix exolicatly Towards that end 
consider the transform tion u = l-2u and v = l-2v Substi~ 

tuting for u and v on th right hand side of (36) w obtain 
u 0 V = (l~u V )/2 which is a mere compact expression 
comparoQ to u+v-2uv We can furth r show that 


u Ct) V ^ w = (l-u v w )/2 wh re w = l-*2w 
m m 

In g n(=ral w'^ have 2 c x = (1 - % )/2 where denotes 

1=1 1=1 


summation in GF(2) and c^ e GF(2) 1 = 1 2 

and = l'-2Xj^ Now for c x we have 


m 


^1^ = 2 


( 1 - 


m a 


ij 


3=1 


) 1 = 12 


13. 


m 


If 

0 

II 

0 

H 

if 

a " 1 
^13 
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NotG that the representation for a^>( clearly shows the 
ffect of a s 

ooing oacl to the minimization problem given by 

T 

(3 5)/; can ignore 1 r for th purpos of minimization as it is 
a const nt The rest of the toims can bo combined as 


T 

min (i Gx r ) 

X - 

Let d = (dj^ dg 

Then u x = ^ 


rc r = l;--2r 

d„) whore d 
n" j 

Th re fore Gx r 


m a 

-]x z j = 1 2 

1=1 ^ 



n 


Thus we have to find 

n nr t ^ 

min (- ^ l^(d r )) = min (- ^ 2 ) v/hore r = l-2r 

With the objective function in this form we can now 
proceed with the r cursive optimization algorithm iscussod in 
Section 341 To round up the for going aiscu sion we give 
thro illust-rntive examples 

Example 3 2 (Hamming Code) Consider the generator matrix G 

1 0 0 0 1 1 0 : 

0 10 0 10 1 

0 0 1 0 0 1 1 

0 O 0 i 1 11. 


given oy 
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T 

Let X = (1 i 1 0) be the transmitted vector Then 

c=(l 11000 0 )^ IS th transmitted code v ctor 

T 

L tr= (0110000) be the received vector For this 
situa ion we have 

'^l " ^1 '^2 ^ ^2 *^3 ^3 ^^4 ^4 ^5 “ ^ 1 ^ 2^4 

d^ = and Oj ~ Z 2 Z 3 Z 4 

Therefor the function to be minimized is given by 

f(^) = “ 5 ^ ^l'“^2~*^3^^4~^1^2^4^^1^3^4'*'^2^3^4^ 

Negl cting l /2 we can write f(z) as 

f(z) = Zj^{“l-Z 2 Z 4 -Z 2 Z 4 ) ~Z 2 “Z 2 +Z^+Z 2 Z 3 Z 4 = 

^l^^l ^2 ^3 ^4^ ^ 1^^2 ^3 ^4^ 

\^inimization of Uj^ with r spect to Zj^ for fixed Z 2 z^ and 


giv s the following table for stage 1 

Stage 1 Stage 2 



~1 

-1 

-1 

1 

“3 

-1 

-1 

-1 

-3 

-1 

-1 

1 

1 

-1 

-1 

1 

1 

-3 

-1 

1 

-1 

1 

-1 

1 

"1 

-1 

-3 

“1 

1 

1 

1 

-1 

1 

1 

1 

-3 

1 

-1 

-1 

1 

-1 

— . 



— 


1 

1 

1 

1 

~1 





1 

1 

-1 

-1 

-1 





1 

1 

1 

1 

-3 
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In terms of Uj^ we can write f ( z) as f(^) = ^ 1(^2 1 ^) + 

Z2(1”Z'2Z^) + z^^z^ 


Minimization of f(z) with resp ct to Z 2 gives the table for 

* 

stage 2 Note th t U 2 is independent o-^ z^ and IheT-efore 
f( 2 ) = -j+z^-z^ Minimizing finally with respect to z^ and z^ 
we obtain 

f(l) = when Zg = -1 and z^ = 1 which on back 
substitution gives Z 2 = -1 and - ~1 Therefore we obtain 
(1110) as the recovered data vector 

Example 3 3 (Reed ^Auller Cod ) Consider the generator matrix 
given by 

11111111“ 

01010101 
0011001 1 
00001 11 1 

Let X = (1 1 1 1) be the transmitted vector 

c = (1 0 0 1 0*^ 1 1 O^) and r = (1 0 0 1 1 1 0 O)^ 



In this case the function to be minimized is given by 


2 




^1^3 ^1 ^2 ^3^4^ 


f( 2 ) can b v^rittcn as 

"^2 / X 
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For this casG the table obtained for the first stage is shown 
below 


tago 1 


"1 

^3 

^4 

* 

^2 

^2 

1 

-1 

~1 

-1 

-2 

-1 

-1 

1 


0 

-1 

1 

-1 

1 

-1 

-1 

1 

1 

0 

0 

1 

-1 

-1 

1 

-2 

1 

-1 

1 

0 

0 

1 

1 

-1 

0 

0 

1 

1 

1 

0 

0 

N ote 

that 

in tho 

first 

stage 


U^Czj^ Z3 Z4) = min 2^ 

^2 


Zj^ also can be consid red as tho f rst stage variable 

From the tabl it is not difficult to s e that z^ = 
Substituting Z2 m f(^) we obtain 

1 

^2^ ^1 ^3 ^4^ ~ 2 (l-Zg+z^-z^z^ ) + 2 

Tabl for th second stag minimization is given below 
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otago 2 



From the table it is easy to see that Zj^ = Substituting 

for in ^2^^! ^3 ^4^ obtain 

Therefore we obtain ~ 1 = 0 Z 2 = 0 Zj^ = -1 

where 0 d notes eitner 1 or -1 Thus the maximum likelihood 
solutions are 

(-1 1 -1 1) and (-1 -1 -1 1) 

The corresponding 0 1 vectors are (1010) and (1111) 

Thus although the transmitted vector is (1 1 1 1) we obtain 
( 1010 ) also as a solu ion This is because two errors wer^ 
introduced in the code vector at the places indicated by * 
in _c The sane received vector r can be obtained by introducing 
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two rrors in the locations 2 and 4 of the code vector 
obtained by (l 0 1 0) as shown below 

Cj^=(ll 00 110 0)^ 


Example 3 4 Con ider the generator matrix given by 



110 111 
0 0 110 1 
0 0 0 0 1 1 
0 0 0 0 0 0 
0 0 0 0 0 0 
0 0 0 0 0 0 


oooooooooo 

1100000000 

0111000000 

1101110000 

0011011100 

0000110111 


T 

s can be obtained by reading columnwise in G and 
associating s whenever 1 s are encountered For instance 

w obtain by noting that the last column contains 1 in 

the z^th position Therefore d^^^ = The objective fun- 

ction IS then given by 


f(zj_ Z2 



1 1 

^ ^ ^ Z ^ Z 1 Z^~i“ Z 1 Z^"* 


Z4- Z2 7 ^ Z4- Z3 Z^ Z3 24 Z^ - Z4 Z^- Z4 Z^ Z Z5 - 

T 

where we assume that the transmitted vector is (0 1 0 0 1 0) 
c = (o"^ 0 1 1 u 1^ 1 1 1 1* 0 1 i 1 0 O)^ 
and r = (1 0 1 1 0 0 1 1 1 0 0,1 1 1 0 0) 


Note that 3 errors occur in this case 
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Now carrying out the steps of the recursive algorithm we 
obtain th following tables 


Stage 

1 



Stage 2 




^3 

^1 

^1 

^3 ^4 

^2 

^2 

1 

1 

1 

1 

1 1 

1 

-4 

1 

-1 

-1 

-3 

1 -1 

1 

-2 

- 1 

1 

1 

1 

-1 1 

0 

-2 

-1 

-1 

1 

-1 

-1 -1 

1 

-2 

U3. = : 

Zi(z 2 - 

'Z 3 -Z 2 : 

Z3) 

U2 = 

V ^ 

)+Ui 

In stage 2 

it IS 

easy to note 

that Z 2 = ‘■^ 3^4 

if 

we put 0=1 

Stage 

3 



Stage 4 



^4 


* 

"3 

* 

^3 


"4 

^4 

1 

1 

-1 


1 1 

-1 

-4 

1 

-1 

1 

-6 

1 -1 

1 

-6 

-1 

1 


-2 

-1 1 

1 

-6 

-1 

-1 

>^2 

2 

-1 -1 

1 

~6 


^4 - 4 ^ 6”^ ^5 ^ 6^ ■''^3 


U 3 - Z3(z5+Z4Z5)+U2 
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if 0^=1 and 02 = 

Stage 5 


^6 ^5 ^5 

1 -1 -8 

-1 0 6 

Tracing back through the tables w-^ obtain 

^6 = ^5 = -1 ^ 1 ^3 = ^ ^2 = -^ 4 ^ 

Therefore th maximum likelihood solution vector is 
(0 1 C 0 1 0)‘ which IS thG transmitted data vector 


^6 = -2^6 '•'J5 
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CHAPTEP 4 

DhCIolON FhCDBACK DECOMVOLUTtOK SCHC lES 

In the previous chapter \ie derived a M.L RDA whose 
computational complexity and stor ge requirements grov/ linearly 
\/ith the block length and exponential!/ with th channel memory 
We shov^ed that attempts to further alleviate computational ard 
storage requirements in th binary ca e lead to minimization 
of pseudo Boolean func ons which in genercl is a rather 
difficult problem The computational and storage requirements 
how ver can be reduceo if one is willing to accept suboptimal 
ch mos Towards this end in this chapter we con ider the 
possibility of incorporating decision ecdback idea for 
recov ring data in BDi systems \Io propose and analyse four 
eecision feedback data recovery (DF-DR) schemes All these 
s lemes are based centrally on the idea of nonlinear Gaussian 
olinination method described below 

Consider the received vector r given by (27) 

r=Hx+w (41) 

Under the assumption ox r al valued data and white Gaussian 
noise the ML data voctor is obtained by solvina 


h’^Hx = I’^r 


(4 2 ) 
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One method of solving (42) is to use the Gaussian Glimina- 
tion (GE) algorithm which consists of for//ard elimination proce- 
Jure (in which H rl is roducod to an upper triangular matrix) and 
back suostitution (in which the solution is obxained by recursi 
vely sol/ing th resulting set of eouations) Now we can incor 
poratc the discr-^to nature o data symbols into GE algorithm in 
the folio ;ing n nner Vhile back suostiiutin instead of using 
the calculated valu s darectly we can us the value obtained by 
passing the calculated valu th ough a nonlinear d cision 
device which assigns an appropri«>tG valu from the finite 
alphabet Since this proc dure xn /oive in reduction of a non 
linear device between t le fo v/ard elimination and back substi- 
tu ion stages the method s callod nonlin ar Gaussian olimxna- 
lon 

This chapter is organa7ed as follows In S ction 4 1 we 
us Cnolesky decomposition t chnique co derive a DF-DR scheme 
which does not tn c noise variance into account oec tion 4 2 
gives a DF-^DR sch no which minimizes moar s juaro error between 
tic actual data and its estimat'^^ A numcricall/ stable DF DR 

SCI mo IS derived in Section 4 3 This cl enc is based on the 

minimum property of Fourier expansion in unite Jimensional 
Hilbert spaces Xn Section 44a DF-DR sch m is derived by 
modifying Austin s [13] assump ions regarding decision feedback 
equalizer for CSDT systems Ir S ction 4 o show that the 
modified jHUstin s oproach is a3 so ossen inl^y equivalent to a 
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nonlinear Gaussian elimination method In S ction 4 6 ai© list 
some observa ions on the methooology used in this chapt r for 
motivating the idea of decision feedback 

4 1 DnCI ION FEEDBACK SGHEivlt USINu CHOLbSKY DECOMPOSITION 

Gholesky cecompo ition technique [74] deals with decompo- 
sition o positive definite matric s into product of nonsingular 
lover anJ upper triangulai matrices If V denotes a lower tri- 
angular matrix /io can write as 

= v’^V (4 3) 

Using (43) (4 2) can be written as 

v'^x = V”^H^r 


This IS equivalent to the forward elimim ion stoo of GE algo- 

T 

r thm Since V is an upper triangular matrix estimate of the 
last elonent of x is obtained as 





c /v 


1 T 

where is the mth elemert of c = 7 ri r and is the 

T 

1 h diagonal cl ment of V Now let be the value obtained 
by passing through a decision device use instead of 

Xj^ for calculating ^^^ 2 . by 


ro 

X 




'm-1 


V 


m 1 




■1 m-1 
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Now 1 v; 11 be t r re ov red data d cide \ on the basis of 
m-i 

Using ^ ano "^n-a cilculatec This proc dure 

IS continued till Xj^ is obtained 

In the foregoing apiroach la t data o th block is re 
recover d first her as in h c s of dcci ion f dback data 
r cov ry scl nes fo-" CoLiT first data is recovered first 
U hough v;hich da a value is recover q first i imm teriai 
in t 0 case o BDT for the sale of uniformity we incorporate 
thi feature of C3DT systems in th folio wino manner 

can also be wiitten as a product of upper and lower 
triangular tvatrices That is 

h'^H = U^U (4 4) 


where U is the upper triang ilar matrix One method o 
obtaining U is as -^ollows 

Let A„ = h'^H and ^ 

Th n and can be written as 



.T ' 


T 


®inm ^n-1 


y £ni-i 

A 


and U,- = 


m 


n ! 



^m-1 Si-1 


1 0 u„ , 

m-1 


Then 




2 T 


1 


T r 

rn'^i 


Un-AVl 
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Therefore y = and d^.l ' U„ lilj,.! 

Therefore 

Note that y is real because 

= ciet(Aj^) 

and Aj^ 3.S a positive definite matrix In deriving the recursive 

form for U we have folloved the method given in [74] (see page 
423) 

In terms of U /e have to sol/e 

U^x = U"^H^r ( V 5) 

where is a low r triangular matrix The procedure of 
recovering data begins 'Vith the estimation of first oata 
] ich IS given as 


-1 T 

where is the first element of z = U Hr is obtained by 

passing Xj^ through the nonlinear decision device Then X 2 is 
obtained as 






U2i«i)/U22 


We continue this procedure till x^^ is oDtained Figure 4 1 
gives a block schematic of the data recovery scheme Jas d on the 

procedure just nov; outlined In the figu e u^ denotes the ith 

•*"1 T 

row o-^ U and v^ d<^notes ith rov'/ of U -D wh ro D = diag(u , 

Uj^) Although u^ and are shown stored in separate register 
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FIG 4 1 Block diagram of the data recovery scheme 
for BDT using Cholesky decomposition 
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banks in the figure in ail a sot of tn registers of length 
ni are sufficient 


Processing of each block consists of m cyci s In the 
figure we have shown the status of various registers for the 
kth data recovery For the next Jata takes the 

place of Vj^ and take the place of Uj^ Uj^j^ becomes 

'^k+l k+1 cycles the in tial state is restored 

and the processor is ready to d al vath Iho next block The 
inner product computer shown is the major computation block in 
the scheme. In the case of binary data tl inner product 
computei used in the feedoack loop ooes not require multipli 
cations 


Computational requirements of OF DP sch ti for BDT using 
Chole ky decomposition method is sarae as that of DF~DR scheie 
for CSDT if the block leng h is equal to the number of forvard 
tap On th other hand the scheme shown in Fic 4 1 requires 
more storage compared to DF-JR schemes for CSDT syst ms How- 
ver with the present day t'^chnology storage requirement is 
not a crucial factor Whereas long bursts of error in tho 
ense of CSDT due to error propagation especially in the 
medium S-JR range is definitely a matter of concern In the 
case of DF“DP scheme for BDT systems error propagation is 
restricted to a block length in the worst situation 
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-analytical e/aluation of the performance of DF-DR schemes 
IS di ficult due to the presence of nonlinear decision device 
However some idea of p rformanc can be obtained under the 
assumption of error-free feedback In this case probability of 

ith data symbol being in error (PER)^ is given by 

CO - 1 y2 

(PER)^ J 1/2 e ^ dy = Q(f^oNR)^) (4 6) 

(SN^)^ 

wher 2 , 2 

q(x) = ® ^ 

X 

(Sr l) can be computed as follows From (4 b) we see that the 

JU 

effecti/e noise vector is given by 
w = U"^h’^w 

Therefore SMP associated with ith data symbol is given by 

(SNR)^ = u^^/(l|u^|P No/2) 

2 JL T" 

wher I II denotes squared length of the ith row of 

and is the noiso variance 

4 2 WSE DECISION FEEDBACK DATA RECOVER/ SCHu E 

In the previous section we described one method of 
obtaining the parameters of DF-DR sch me using Cholesky decotn- 
po ition in which we did not take noise variance into account 
We now derive a DR schem cased on m ii num mean square 
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error (ivii^SE) criterion This scheme accounts for the noise 
variance 

Ve first define several vec ors and matrices which will be 
used in the following derivation 
T 

b = H^r 

Xc = ('^l ^2 1^^ 

''k = '''k 1 ''k 2 ''k k-P^ 

'^k “ k “k 

Hj^ = H with first 1 1 olumns removed 
hj^ = kth column of H 

= H With first k-1 columns 

The objective is to obtain a set of coefficients Uj^ ^ 

'^k k+l “k m Vki V |,2 such thut estimate 

Of the kth data symbol given by 

^k “ k '^k m *^-n ■" ^k 1 ^1 “'^k k~l ^k-1 

(4 7) 

is closest to th actual data Xj^ in tne mean square sense 
That IS 

n 

MSEj^ = ] IS minimum 

Here E[ ] denotes the standard exp ctation operator Using 
th principle of orthogonality in v'lSE minimization [7] wc 
ootain 
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“0 j = X x+1 m 

and i-[e^ ^ i - 1 2 k 1 

where ej^ = Using the fact that 

we get 


Hx + w 

J J 


E[ej^b^] = E[(ujHjhx + u^Hj^w - VjJxj^ Xj^)(h^Hx + Injw)] 

T' T' T' T* €\yT' "T* /*v ‘T* ’T' 

= Uj^Hj^HH hj „ hjh^ + ujdjhj = 0 


.TuT., 


(4 8) 


j = k k+1 

Similarly cCej,x^] =0 gi /es 

= 0 1 = 12 


m 


TuTu 

ak”ki!i 


V 


k 1 


ki 


m 

^ E u 

J=k 


kj “j-i 


k-1 


’’I 


In th matrix vector notation 


''k = Vk“k 


(4 9) 


Substituting (49) in (4 8) and rearranoing terms we obtain 

N 


Uk = [hJ[HhT + 5^ I - Hjhi^ 

= [hX + 1 ° I]-" (hX)-' Hr-i, 

But (hJ Hjhj, = (1 U 0 


\"k 

o)T 


(4 10) 


8U 


Therefore Uj^ is obtained as the first column of 


[hX 



I] 


1 


Fin lly the resulting MSt associated with the kth data is given 
by 

_ ~ m 

. 1 1 T A l 


- 1 iikVk 1 1 V 


o 


= 1 


^j'^k j+k 


(4 11) 


The irean square error given by (4 il) reflects to a great 
extent the quality of ne DR scheme It an be consid'^red as 
a mea ure of the performance for it indicates the variance of 
the error If MSE is significant probability of the kth data 
symbol being in error is large Thus i/(WSB)j^ may be considered 
as a measure of effective SIIR associa ed with the kth data 
symbol 

Comoonents of u. can be computed recursively u ing Trench 

T 

algorithm [75] because the matrix iJ is a positive 

definit Toeplitz matrix First /ve cal ulate given by 


T 

= u„^ = (h^H„ ^ I) 

m mm mm Z 


wh'^re hIiL = hlh_ = h^ -r h? + 
m m '“til' mol 


-1 


g 


The oth r Uj^ s k “ n-1 m~2 1 can be recursively 

obtained by following the procedure given in [75] Thus the 

O 

computational compiexi+y is 0(m ) as the procedure for 
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calculating Uj V = 1 2 m is same as inverting a mxm 

positive def nite Treplitz matrix by Trench algorithm The 
implementation scheme is same as the one shown in Tig 4 1 
except that and v^ are given by (4 9) and (4 10) res- 
pectively 

4 d DECIqIGH FELDB/^CK SCHl E based 0^ hlNIMU PPOPERTY 
OF FOURIEf eXPAMSIONS 

So far in our analysis no thought has been given to the 

problem of illconditioning oi the channel aatocorrelation 
T T 

matrix H H H H i said to be illcond t: oned if the ratio 

T 

of maximum ard minimum eigenvalue is large [74] When H n 

T 

is illconditioned the solution obtained oy inverting H H is 
v ly sensitive to truncation errors and noiso in the received 
vector It IS therefor'^ essential to consider numerically 
stable methocs Towards this end in this section a numeri- 
cally stable method of solving linear system of equat ons 
ba eel on minimum property of Fourier expansion due to 
/ang [58] is dopted to derive a decision feedback data 
recovery cheme 

Minimum Property of Fourier Expansion For an/ finite or 
infinite linearly independen sequence of el ments 
of a Hilbert sp^ce there always exists an orthonormal 
system ^2 such that 

= “jdlSjll 3 = 12 
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where |3k)?lc/ I I <4 12) 


1 2 


This procedure is cnown as Grain~Schmidt orthonormalization 
Here (a p) denote inner product of a and p 

Consider an eienent a m Sj_ Then x s v/hich minimize 

3 

the norm 

11“ ~ 

j=l ^ 


are hnovm as the Fourier coefficien s of a They are given by 
"j = (« Pj) 

This property of s is known as the minimum property of 
Fourier exparsion 


Now let Z 2 be the m column vectors of H in 

(4 1) Since the column of H are linearly independent ortho- 
nor Tiali zing Zj^ ^2 obtain Then 

we have the rollowing result due to V/ang [58] 


Theorem 4 1 Let r given in (4 '' be an el ment of an n- 

diment3onal Hilb rt space x s which minimizie the norm 
m 

\\^ i* are given by 

j=i 


X. 


m 




1S> 

X 


^ 3 


= (r 


j = 1 2 m-1 




2 


(4 13) 
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(Proof of this theorem is not given in [58j Therefore for 
the sake of completeness we prove the theorem nn the following 
manner ) 

Proof We knoN that given r and s the c, s which ninimize 

3 J 

the norm 

m 

j|r - S c i I are given by 
J=1 ^ 

Cj = (r Yj)/IU3ll^ 


By substituting for in terms of z s using (4 12) and 

J 

grouping the terms common to s we can obtain the recursive 
equations given by (4 13) To see this consider 
ra 

S C.X-, ai^d substitute for using (4 12) This gives 

j_l j j j 


m 

£ 

J=1 





iiiTTF 


(Z3 i2>X2, ^ 


+ c„( z„ 
m'^m 



^ -m ^m~l^^m~ l^ 

1 i^m-l ^ 


From the above expression it is clear that the coefficient of 
^m Si 
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The coefficient of gives 

Xm-l = 

The rest of the coefficien s can be simiLarJy obt ined 

M. 

It may be noted that in th method discus ed above is 

m 

ootained first which is used in ootaining hy back substi~ 

tution However for the sake o-^ uniformity we wish to obtain 
first This can be easily don by orthonornalizing the 
columrs of H starting from the mth column Le y 2 

denote the orthogonal columns ob am d this way Then the 

aoverning equations of the data recovery s heme are given by 

^ = (r - Y \\ ik)/llyill^ (4 14) 

1 k* — 1 

i = 2 3 m 

where hj^ is the kth column ofH 'y^i = 12 m are the 

Gs mates of the data values obtained by pa sing i = 1 tn 
through a nonlinear decision device 

Figure 4 2 shows he data recovery sch me Proces'^ing of 
each block of data consists of m cycles lajor computational 
load IS due to the inner product cDinputation in every cycle 



FIG 4 2 Block diagram of the data recovery scheme using minimum 
property of Fourier expansion 
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Tne perfornance of the sch me can be analytically 
evaluated in th absence of error The probabilit/ of error 
associated with ith da ct symbol is jiven by (4 6) The oNR 
associat d w th the ith data symbol can be calculated as 
follows Substa Luting for r from (4 1) in 14) for v/e 
qe+ 

= (Hx H w Li)/\ IyiI 

m ^ 

= ( ^ + (w 

j “1 

Substituting for h in t rras of y; using (4 15) we see that 

"" ^1 I /ll 

Similar expressions are obtained in the ca o of other data 
symbols al o Thus 

(SNR)^ = llij.lP/lNo/2) 

i ly^j 1^ 1=12 m rhus denotes a m asure of the perfor- 

mance of the seneme anc it is related to the c annel sample 
r soonsG vector V^hen the columns of the channel convolutional 

matrix H are orthooonal jjy^l P = 1 for i = 1 2 m if 

T 2 

h^h = 1 'Vhen corr lations exist among ho columns < 1 

1=12 m This can be seen by noting that 
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m r) 

. iil )Vk/l \^k\r 1 = 12 m-l 

(4 15) 

Fron (4 15) we obtain i and 

I J ^ '^=1-1+1 1^ < 1 

4 4 DuCISIOM FbCDBACK SCHufte BASlD ON ^ODIFICD AUSTIN S 
mPP POACH 

On of the most vadely used DR schene n the context of 
CSDT s/ tem is the decision feedback equali or Austin [13] 
has deriv d the equations of decision feedback equalizer for 
C DT sys bens by making suitable assumptions so that detection 
theory results can be easily applied A^ a counterpart of this 
scheme in this section w derive a DF~DR schene by modifying 
Austin s assumptions In the following discussion we assume 
binary data transmission 

Let denote the symbol on which decision is to be made 
Vi/e modify Austin s assumption in the following manner for the 
case of BDT 

1 Data s /mbols are independent Gaus lan variables with zerO’- 
mean and unit variance for i > k 

2 are known for i < k 

3 Noi e components ar independent Gaussian variables with 
zero mean and variance Nq/2 

4 Xj^ ta<es 1 or -1 value with equal piobability 
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Th 1 kelihood ratxo for Xj^ rs given by 

L = 

^ p[r/x^ Xj^=~l] 

wher x^ = (x^ x^ ^Ic ^ ^^k+1 ^k+2 \n^'*' 

denote the future data value with resocct to Xj^ p( ) denotes 
probabili y den it/ function won can oo alternatively 
expressed as 

L lLlk.lt'4LllL’=<k=i 

/p(r/xf, xf p(xl) dx( 

flaking use of the a surrptions made xn the beginning of 
this s ction VC can v/riic 

- 1 n n 

P(£/2<k -^k ^k^ "" ^1 ^1*^1 

where is a constant and 

^ = h ® h ^j-rL ^ ”■ <4 

o j ==o 

^1 = I” ^ \ h 1 = 01 m-l (4 18) 

o J =0 

Dumping all the xactors which cancel out v hile forming Lj^ into 
we re /rite (4 16) as 
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- m k-1 m 

P(r/x^ xf Xj^) = K 2 exp[xjb + E 2 I <E 

1 “k+i 1 ~i j — k +1 


k-1 m 

- 2Xk x^a^., - 2x, x^a,.^ 


m m 

i=k+l j=k+l ^ J ^ J 


Using assumption (1) we also have 


-r 1 ^ 

p(x?} = exp[ i ^ 


n m 

E X X 6 ] 

1 *1 ‘-'-I t J 


where 6 


ij 


i-k-hl j=k-l " ^ 
1 if 1 = 3 

0 if 1 ^ j 


U mg the structure of multidimensional Gaussian density fun“ 
ction and following Austin we cm obtain the numerator of Lj^ 
as 


exp[X|^bj^ - 2x, 


- ^k^3-k 


k~l 

£ 

1=1 

) 


m 

E 


k + E 2 (| b 

^ j=k+l i=k+l ^ J 


m 

E 


k 1 

2 x^a 


n~l 


n '^3 -n 


k /I u 
‘’01^2 ^ 


k-1 

S 

n=l 


^n\-n"^k®i-k 


)] 


k •*’1 

where q are the elements of m trixv elements of 

3 1 

ar given by 




6 


ij 


1 0 > k 


(4 19) 
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Thus we obtain 


k— 1 m -1 -j 

^ 1=1^^*^ n=k+l 3 =k+l 2 J 


k-1 




m 


Hov/ let gu = ” q!^- a . n = 1 2 

i=k+l "■ ^ 

3 = 1 Hi 


ra 1 (4 20) 

m 


kj k 


m 

I 

i=k+l 


and f,.. = ?a. ,. + 2 £ k = 2 3 


0=12 


m (4 21) 
k-1 


The decision ruJo in terms oi: s ard s become 


m 


3:=k+l 


k>-l 

S 

=1 


x^=l 

X j^”“l 


0 


(4 22) 


k = 1 2 


m 


Th decision rule given by (4 22) clearly reflects the 
structure of the data recov'ery schen The structure is shown 


9k n) 


in 

Fig 4 3 

in the figure 


b = 

^•^1 ^2 

^ 9k = 

^^k k+1 

^k= 

^^kl ^k2 

^k m-1^ \ 

^^k+1 






Buf 

St( 


the data recovery scl 
jstms approach 
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In order to lal^e dec s on regarding the I'th d ta symbol 
v; need the nowledge of gj ^ j = k+l n and f 
j = 1 2 V-l 

Ue illustrate th for going discu sion with an example 


c amole 4 1 Consider a > ock contaning thr data values 
X = (x^ X2 ><3)^ transmitt d over a chann i having (h^ hj^ h2) 
a its finite impulse -response s quence Then or a received 
vector r = (r^^ £2 ^5)^ have 

o o 


aj_ = r ^ 


(r^h^ h r^h, r^h2) 


^^2 - \r ^^2"o " " 3"1 
o 


IJ 

o 


■=3 = ,r ( * r^hj) 
o 


^0+ 5 


The rntrices R. k = 1 2 
1 ' 

^1 


^ 1 
ao + 2 


a'f' given jo 

R2 = [a^ |] 


Lot 




^22 

"'2o 

^32 

^33 


ana 


^ "* ^33 


Then 
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923 "" ”^33®i ^21 ^ 2ai 1-292332 

^12 “ '’^922^1''''^23®2^ 

9i3 " "'^932^ I*** 93 3 ^2^ 

The decision rul 22) 

^12^2 

*^2 ^ ^23^3 - ^21^1 

^3 ^31^1 ” ^32^2 

4 1 Coiputation of the Tap Coc ficients 

ihe CO ffici^nts j = k+1 m c n le vibnqo as the 

o coof icients of thp forward fxl er in he cas f Xj^ More- 
ov r these co fficient can be calculated ef'^ici-^ntly using 
Levinson-Durbin algorithm [7 ] i or that pu pose (1 20) is 
r written below in the matrix vector forn 

Sk = - ^k-^k 
k k+1 




^32 "" ^®1 


becomes 


Xj_ 1 
Xj ^=-1 

X =1 
? 

X 2 =~l 

X 3 -I 

X 3 = 1 


0 


0 


/here Cjj^ = (o 


gn,) and a, = (a^ ag 


®m-k) 
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IS the inverse of Pj^ we can v;rite gj^ as the solutD on of 

"" ““^k 

v^rhere Rj^ is defined by (4 19) 

Rj^ IS a symnetric Toeplitz matrix Thereto ^e can make 
use of Levins on-DuT'bii recur i/e algorithm given in [7] for 
obtaining s The required st ps for computing gj^ s are 
civen below 


1 

^11 

— ( 

o 2 0 

1 

2 


2 

^1 

= (a, - 

1-1 

1 d , a ) 

1-1 J 1-3' 



3 


= 

\ji -1 

11 1-1 1 J 

1 < 3 

< 1-1 





1=2 

3 k 

4 

®l 

= (1 

*^11^ 1-1 



j 

^n-1 m 





u 

^k k+i 

= '^a-l 

1 1 = 1 4 

m ' 

k = 1 2 


442 Modifieo Au tin s Approach and Nonlinear uaussian 
limina ion I thod 

In this seer on we sho/ that the modified Austin s 
approach may be vi wed as nonlinear Gaussian elimination method 
•fith approoriate odificitions For hi purpose 22) may be 
written using ma rix vector no ation a 


9d 



ub ~ rx < 0 (4 23) 


where 0 is a null vector and ^ Ipnotes symbolically the non- 
linear d cision de/ic Uote that th riobt hand sice of (4 '>2) 
IS an esti late of Xj^ Therefore th right hanu side of ( ^ 23) 

1 an estDi at of tio data vector x That is 

Gb - Fx = I,flX 

L t = F T 1 ^ Th n the above equation can be wri ten as 

b r X = b 

IS invertible since the diagonal elements ar nonzero) 

r 

uut b = H r/(i'jQ/2) which gives 

X = H^r (4 24) 

* o 

2 

Conoarinj 24) v/iti (4 2) it can be seen that C V plays the 

T 2 "“i ^ 

role of H ll Thus w mav call j^—G F as eff c ive autocorrela- 

tion natiix With ihis int rpr tation \ sef^ that modified 

Aus in s approach is equivalent to nonlinear Gaussian elimination 


method 
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t 5 SO IE OBSnPV TIONS 

As we arc dealing with the problem of sol /no lin ar equa- 
ors in the presence of noise when the unknown variables take 
VI u s from a finite set it is natural to think tha availaole 
methods for sol/im linear equations must apply with approori te 
modifications Vi/e demonstrated this b considering four DF~DR 
sche nes which are centrally oased on tl dea of nonlin ar 
Gaussian elimination method 

The two DF-DR chcmes considered in Section -^1 nd 
S ction 4 3 essentially modify already existing me 1 o for 
solving linedr equations by incoiporatin j J cision feedback as 
explained in he beg iing of the chapter However the methods 
discussed in o ctions 4 2 on J t 4 re den /ed based on a set of 
as unpt ons and a p-^^rfornance criterion DF DR schene of 
S ction ^ 2 emoloys math^m tie ally tractabl IMSE criterion 
while in Section 4 4 v;e used more difficult e on eory 
approach In this approach he complexity of the scheme is 
lleviated considerably by assuming that the data value are 
indep ndent G ussian variables The major ide-rlymo assumption 
in both ihe sch m s is that the past decisio s are rro ree 

In the absence of noise all the four sch les no aouot 
provice error free perforiance even wirhou th nonlinear 
decision device hoi/ far d c sion f edbacl nolps to improve 
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the perfomance ir the pres nee of noisp* can only be evaluated 
vjith the help of sinulation Although decision edback is 
an old idea [77] no useful mathenatic 1 analysis is ava-J able 
;hich deals with analytical evaluation of perfornance The 
rnajo tunblng block appears o he th nonlin nr decision 
evico \/hich d fies y analysis In Chapt r P //e gi /e Simu- 
la ion resuJ ts on DF OF scho les u vrlooed in oectiors 4 3 ard 
4 4 
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CHaPTi- 5 

DATA R COVi'RY SCIEfix-o USINb k Cl CULANT C0MPLETI0I13 

In he previous chapter we maoe use of past decisions and 

band d foeplitz natuie of rbe c lannol autocorrelation matrix 
T 

H H for deriving decision f edback deconvolution schemes v/hich 

are suboptimal compared to t tL dc onvolution scheme of 

Chapter 3 It may be oos xved ha so far in our study no 

attention has been given to the convolution matrix H of the 

channel Ve shall see in this chap er •**hat H has some int rest- 

T 

ng propertie compnred to i- r' from the point of vxew of 
-'educing computations and storag requir ments Ba ed on k- 
irculant conoletion of F we develop two data lecovery schemes 
These schemes are easily implemen table and have reduced comouta 
tional complexity furthermore uilike the DF DP schemes of 
Chap cer 4 which are nonlinear the DR schetes studied in this 
chapter are linear an they permit analytical evaluation of 
lor performance 

The chapt r is organiz d as follows In Section o 1 we 
1 reduce the notion of k-C rculant co oletion and describe its 
application to deconvoluti :in Tv o netbods of inverting l- 
Tirculant matric s are gi /er in Section o 2 The -^iist method 
us s FFT for inv rting k-f i culant com letion of H Complexity 
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of his al'aOrithF is 0 (nlog 2 n) jvh re n is chc nunber of 
rows of H Th second merhod makes use of o rsit/ nd block 
k~rirculant nature of the K-Circulan compl tion of -i to pro~ 
yiJ an OCnlog^Qj^) algorithii \;hore g^^ = g 1 In S ction 5 3 
we discu s two DP schc n s which are baseo oi k-DFI Jonain 
d convolution Jiscjssed in See ion 5 1 30 *^ direct inversion of 
c Circulant completion of ri d scribed in c ion b 2 In 
Section 5 4 V/ obtain c/plicii e pression for probability 0 
rror and compare it with rho pcrfornance x DF scheme cased 
on least quare d convolution net) od 

5 1 k-CI-iCULMNT COHPLuTiOd AID Fit iTE DECON yOLUTION 

r onsider the received vector r given oy (2 7) 

r=Hx+w (cl) 

/hen no statistical charaetc ization of x and w are available 
je know that the maximum likelihood ostirnat of x under the 
assunotion of r al valued daca is giv n by 

Xl = H^r 

The riost time con uning 00 ration in oat niig Xj^g is the 

T T 1 T 

in / r ion of p H Ev n ix we assume that (4^^-) is computed 

before the transmission of data memory requirements can be 
como prohibitive Therefore even vath the availabili y of 
s voral fnst algorithms for inverting [70] [ 8 O] there is 
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still need or processing algori hns th + arf» easily implemen 
table In this s<=ction le consider the apolic xioi ox k 
Cnrculant matrices [81] [82] •f’or oivinc deconyoiu on pro- 

bl rn i or he sake ox sy reference so le of the results 
r garding k~v^irculant na rices are given in ppendiy D 

As an illustration vie consider (o 1'' and it 1 -Circulant 
count^roart forn=5 m=o and g = 2 In this case (o 1) 
can be written as 


* r 


h 

0 

0 ■ 



** W * 


r 

0 

0 

kho 

kh. 




* 

0 


0 





0 


0 



2 

1 


0 


0 

^1 




0 


^0 


Wi 


hi 


0 

C 

kh2 


"i 



^2 


^2 

hi 

ho 


^1 


^^2 


'^ 5 . 

hi 

ho 

0 

0 


^2 

4 - 

W2 

^3 

i 

0 

h2 

hi 


^2_ 




L 

h2 

h 


0 


0 


W3 

^4i 


• 0 

0 






0 

L 

u 

h2 

4 



0 


W4 


r=H^^+w (52) 


where Hj^ i the k-Circulant completion of 4 and is obtained 
by padding x with tA^o zeros Ix i not d3 icult to see that 
the T oresentation giv n in (5 2) in terns of H|^. is also valid 
in the genexctl case v/hen r is fini e (n = tntg) ano x is padded 

with n~n zeros to obtain x„ 

© 

From row on we call the Circulant conpletion of H as 
it IS easy to sc'^ from (5 2) that is a Circulant matrix 
Similarly vail denote the Skew Circulant completion of H 
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511 Deconvolution Usin Circulant Comoletion 

Consider the circulant completion of If P X no J 
deno e DirT ol r x and w respcctiv ly using tne prope ty of 
cyclic convolution ['^9j we can write (5 2; as 


R = h X + W (53) 

where H denote the DFT of the first column of H (H X) 
denotes poin wise multiplication of tl compon nts of the 
individual vectors Equation (5 v) suggests an easy way of 
decon olving >Mhcnever none of the component of F is too small 
or zero In su h cas'^s an estimate of x can be obtained as 

^le = F~^(n/H) (5 4) 

where F denotes the UFT matrix of orJer n and (R/H) is a 
vector obtained by componontwis division This method oi 
dccon /olution can be attributed to Cooley [83] who after 
becoming instrumental in th do/elopment of FFT algorithm [S4] 
proposed the forcg:)ing approach as an application of FFT in the 
context of signal process ng Some of the problems associated 
with such straightfor >?ara application to situat ons involving 
erroneous measuT-e nents hav been discu sed t length by Hunt[59] 

5 12 Deconvolution Using k-Circulant Copletion 

In this section we show that (5 3) and (5 4) can also be 
generalized to the cas of k-Circula'i completions For this 
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S 1^ 


purpose let b = (b^ ^ “ ^‘'l ^1 

be two vec ors /e define k-cyc ic convolution o-^ d and c as 
anothe vector d given ly 


1 n-1 

d, = 2 b + k 2 be, v . (5 5) 

1 J =0 J J=a+1 J (i-j)™dn ’ 


/e detine k~DFT of b anJ c as (see Apponoix B) 


n-1 

B.= 2 bf\w)'^ 1=01 

j=o 


n~l 


(5 6 ) 


n-1 

°kl = c (\w^)^ 

J-0 ^ 


1=01 


n-1 


(5 7) 


resoecti\ely corresponcing inverse tran forms are 


1 

b. = •“ 2 B^A XvP)^ 1=01 

^ ^ 3=0 


n-1 


(5 8) 


3=0 

vj! re A is an nth root of 1 nd w 
lollowing theo em 


= e 


n~l 

32Ji/n 


(o 9) 


Now we prove the 


Theorem 5 1 If Bj and C^. represent k~DFT of b and c resp eta. 
/ ly giv n by (5 6) ana (d 7) 


'X 


1-01 


n-1 


(5 10) 


fik - (Bj^ C^) 

whore *^jc IS the k~DFT of d 


(b 11) 
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P roo 
(o o) v/o obtain 


SuDstituting for and fron (o 8) and 


J=0 t=o s=o 


n-1 , n-1 i . I n 1 , 

" ! , n 4 ^ S C-ks(^« ^ 

J *”1 i *C“^0 S ““ o 


Sine x'^ = k on r arro.nging the above expres ion 

. _ 1 y ^ P ^-1 “S j(s-t) 

d - 2 ^ J' t ^ks ^ ^ ^ 

n j=o t=o s=o 


, n~l n-l n-1 n 

+ ^ 2 S 2 B. , C, X"^ w 

n j=i+lt=os=o ‘^® 


Lh nging the order of summation it tccomes 



, n-1 n-1 


n-1 


d = 
1 

-? ^ ®kt 

Ss 2 

X' 


n — o s~o 


3=0 




1 

r 

if 

But 

= 1 

1 



3=0 

1 

L° 

If 


n-1 


Th refore d = E Bj^ ^ i = 0 1 

s=o 


Using (5 11) we easil/ ee tnat if 
k-DFT of h r X and w respectively then 


\ = Hk ^k 


(5 9) in 


i- 3 +n) 


beco nes 


n-1 

T«- 


/ and Wj^ represent 
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From vvhich the deconvoluti ->11 solution can be obtained as 

provided none of the component of are zeio 

513 N ed for Deconvolution by k Girculant Completion 

Note that when som of the DFT co'=>fficients of the 
channel are zero ( cannot be used for deconvolution as 

oivision oy zero is not permitted In sjch cases even if 
IS singular for some other k H mav oe invertible {1 
all the k-DFT coefficients are nonzero) Thus ir some cases 
it ma be raor advanfcag ous to use k DFT ms ead of l-DFT As 
an illustration consider a simr«l example of a channel having 
the following H matri'^ v;hen n = 4 and m = 3 


10 0 


r 

0 

0 

L 


10 0-1 

110 

oil 

Hi = 

110 0 

0 110 

»-l = 

110 0 

0 110 

1 

0 

0 

1 


0 0 11: 


0 0 11 


1 1 IS not diffi ult CO see tha \Circulant completion) is 
singular whereas the Sxew Circulant completion of H is 

nonsingular In -^act det(Hj^) for this channel is l~k There- 
fore IS nonsingular for k 7^ 1 

It may be further noted that computation of k-DFT is as 
simole as 1 -DFT co mutation by FFT To se* this consider 
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K DFT b j von by (5 6) It can bo alternatively wr_tten dS 


3 


ki 


n 1 
h ( b 
3=0 J 


v^hcre A is an nth root of < Conputation of s may be 
arned out by con^jting Fi: i of A s j = 0 1 n-1 

instead of s Th s aspect na cs -DI T application more 
attractive 


5 2 DcCOlIVOLUiION BY DIR CT I VDRSIO 

rh stimate or obta ned via 1 DFT and k~DFT 

gi /en by (5 4) ana (5 12) can als bt obtain d by directly 
inverting whonev r is nonsingular Th refore we can 
ritt 


-ke 



r 


(5 13) 


/e now pre cnt two metiods of inverting 
Me hod 1 

can bo expressed as (see appendix B) 

= D"^ C D 

2 n-1 

where D is a diagon 1 matrix vith 1 a A A as the 

diagonal ntries C is a Circulant matrix v;ith first column 
elements Dh where h is the ci nn 1 inoulse response vector 
Therefore can b obtain d by using FT Then is giA.«Qn 

by 
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= D ^ D 

\,/hen n IS a power of 2 inversion oi C can be carried out 
using 2 nlog 2 n -f n complex arithmetic ooerarions [82] Thex 
fore the comolexity of this method is 0 (nlog 2 n) 

Clements of the first column o are obtained a 

follows L t C, 1 denote firsf colunn l*=‘me ts 

o 1 n-l 

of C ^ If C. C , denotes the first column elements 

0 X n— X 

of then are giv n 1 / = C^/ A ^ i = 0 1 n-l 

\^ethod 2 

In this m thod sparse naiuie of is exploiteo to derive 
an 0(nlog2gfj^) comolexity algorithm for inv rting First 

V illustrate the method for Circulant matrices when n = 
v^htrc gj_ = g+1 Let n = 8 and g = 3 Then th Circulant 
matrix has the following form 
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Note that is a sparse matrix Direct applicntion of FFT 
means comout t on of jFT of the first cPlumn having 4 zeros 
Anoth-^r mor efficient approach is to w ite as 



//hcnever exists Thus v^e obtain 


PJv + J3 = 


PB + QA = 0 

Solving the two equations w obtain 


A+B = (P+Q)'"^ and \-B = 


(5 14) 



Cleail/ P+Q IS a circulanb v;hile P-Q is a Skew Circulant Note 
that both are full matrices (no zero entries) If is a 
power of 2 \je can invert (P-tQ) and (P-q) using 0(nlog2gj^) 


conoutations 
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No j 1 t us consider th^ general case whore divides n 
s-ich thit r = rgj^ (r > 2) /e can partition in this case 


Hi = 


O 

0 

0 


Let exist and 




B. 


O 
/ . 


0 

0 


’r-l 


Br_i Bj.^2 


0 

0 

0 


A / 

1 o 


U :ng the identity we obtain 


A_B + A-i B 1 
00 1 r—1 


= I. 


^o®r--l'^^l®r-2 

A_B o 

o r— 2 1 r—o 




= 0 . 




= 0 . 




as 


(5 15) 
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dy adaing the abov equations we obtain 


( 8 ^ + 0 . 


o 







How consid r rth roots of unity They are 


(5 16) 


1=01 


r-1 v^here v; 


eJ2 A 


^\ultlpylng ith equation in(ul) 0<i< r*>l by and 
adding we can obtain 


(3^ + -r T (5 17) 

X 

Similarly multiplying ith equation in (b 14) by w and adding 
we get xhe general rcl tionship 


r-1 


1 = 0 


^ k = 0 1 


r-1 


(5 18) 


V(/hich contains (5 16) and (5 17) as Particular cases 


Thus in order to determine s we have to forn r k 
Circulanxs (k v;ill be different for each Circulant) whose 
inver ion can le achieved using 0(11109292) computations when 
g^ IS a powei of 2 Finally s are obtain d using (5 13) 
with 0(921092^1^ more computations Thus th'^ to al complexity 
of the f ethod is 0(nlog292^ ^ grov/s linearly v/ith n for 

fixed 92 
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-1 

Extension to the ca e of is straightforward In this 
cas for n = rg^ we can write 



Since inverse of a k-C rculant is also a k-Circulant w 
can partition as 



Thus equation (5 14) becomes 



^^1 ®r-l 


T + 

Ai B o 

= 


1 r'^2 

^1 

A. Bn "h 


= 0 

0 1 

1 o 



(5 19) 
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low let X be an rth root of k Then multiplying ith equation 
in (5 19) 0 < 1 < r~l by ( Xv/)^-^ and adding ;e obtain 

r-1 , 

a = ( i-(xw)’^“^ j = 0 1 2 r-1 

(5 90) 

It JTiay oe no ed that he p ocedur is s nila to the case of 
Circular t matri inversion 

5 3 data PEr'OVERY SCHEhlo 3 -whD ON k-CIRCUL COMPLETIONS 

;Vhen S JR is hijh the 1 -DFT pproach oescrib c in 
woction h 1 2 can be used for i cov ring data Such a scheme 
IS shov/n in Figure 5 1 Although two TFT proce sois are shov;n 
in the figu'^e one is sufficient in actual practice for at a 
tine only one of them will be in use 

Note that in the case of the DP scherte show*" in Fig 5 1 
for every block of data transmitted k-DFT of the received 
vector an i an in/erse DFT i to be performed One way of 
avoiding calculation of DFf for every block separ tely is to 
use the elements directly in the DR schem Such a DR 
sch m IS shown in Fig 5 b This scheme inyolves a circula- 
ting hift register of length n in v/hich fir t row elements 
or are stored and circulv^ted The tai coeffiC3 nts of the 

cyclic transversal filter are the received observations 

^n-1 multiplier with coefficient k in the 

feedback loop of the circulating niter dep nds upon the cotr^le 
tion used 
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Hk 

FIG 5 1 Block diagram of the data recovery scheme 
using k-DFT 



FIG 5 2 Data recovery scheme based on k-Circulant 
completion 
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Processing of each data block consists of m cycles If 
the entire processing is completed in nT seconds v^hexe T is 
the symbol duration th-^re will be no need for long buffers to 
store the incoming blocks In this case the delay involved 
Afili oe one block dux tion Thus two registers of length n axe 
sufficient One is used in the processing whil the other one 
stores observations arriving due to tho next block 

The complexity of the DP scheme shown in Fig 5 1 depends 
on the method used for implementing FFT Since FFT algorithm 
IS parallel and recursive it is suitable for those real time 
applications where cost of the DR schemo is nore important than 
the probability of error performance 

ma 3 or advantage of the DR scheme shown in Fig 5 2 is 
that in its implementation complex arithmetic is totally 
a /Old d Moreover its implementation is simple compared to 
DR schemes considered in the previous chapters 

h 4 PERFORA/uMCE CONSIDERATIONS 

In the context of data transmission it has generally 
o en observed that the complexity of DR schemes seem to be 
Girectly proportional to the desired performance If vel Thus 
minimum probability of error (the best perfo-rmance) demands 
hiQhly complex DR schemes Therefor we may inrcr that DR 
schemes based or k-^Circulart comcle ion being less complex 
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tie r perfornance nay not be satisfactory compared to DP 
ch nes of chapt rs 3 and 4 Hov/ever th*^ DP sch mes con- 
sidered in this chcioter have a distinct advantage over DR 
schen s o veloped in th prev ous chapters when the S IR is 
high In this section we obtain explicit expressions fox 
che probability of error and comoare it with the performance 
0 DR scheme based on least square deconvolu ion Simulation 
studies on the performance of the DP schemes tudied in this 
chapter are presented in Chaoter 8 

To begin with let k = 1 Substituting for r froii (5 1) 
in (51 ) we obtain for k = 1 


fO 




(5 91 ) 


From (5 21) we s e that the effective no3 se variance deiends 
on th elements of Since rhe elements in any row is a 

cyclically shifted ver ion of the first row and since noise 
corponents are statistically independent and re of equal 
variances the effective variance is given by = <^2*^1 
v/here 



n 

S 

1=1 



(5 22) 


and h], is the first column of V/ith th kno\\ledge of the 

2 

DFT coeffici nts of h we can exor ss dj^ alternatively as 

1 


0 1 ^ 

1 ^ 1=1 ^1 1+2 


> 


n+1 


(5 23) 
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Using (5 25) (5 26) and (5 27) v obtain 

9 , ^ o 9 o 
n < n^( Z h^) or 

1=1 1=1 ^ 


n 

Z 

1=1 




Using (5 22) in the above inequality we obtain (5 24) 

V 

(5 22) and (5 24) are also valid in ihe case of k Circulant 
completion since |kj =1 Therefore we can write 




g 

n( L 
1=1 




(5 28) 


It IS interesting to note that all the data symbols recovered 
have the same probability o being in error because SNR asso- 
ciated with each of them is sane It is given by 


(SNP)^ 


,2 ^ 

^k 


1=01 


m-1 (5 29) 


When jk| =1 The probability of error (PER) is given by 


/ 


1 2 

1 “ 5 - y . 

' — - e dy 

.f2i^ 


Q{- 


Ik V(No/2) 


\ r(Ny2) 


) (5 30) 


PER = 
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Let us no// consider th probability of rror performance of 
thG OR scheme based on LS—deconvolution solution given by 
(5 2) In this case SiJR associated with data symbols will be 
different The average orobaoility o error is given by 


PEP 


Lo 


1 

m 


m 

2 Q( f 

1=1 / N, 


(5 31) 


2 0 

vdiere ^ is the variance of the loise compon-an associated 

with ith data symbol /n pper bound for PERj ^2 can be obtained 
as 



where d 


2 

max 


max 

1 < 1 < m 

•ncM 


(5 32) 


2 

We note that dj^^ is the squared sum of the elements of the ith 
row of (H^H)~^H^ Equivalently they are the diagonal elements 
of (H^H)"*^ (because (H^H)"^ = (H^H)'*^) We now 

show that 

d^ > d^^ 1=12 m (5 33) 


DS a particular rase of a more general result Let 
H = [h H 1 where Hq is the souare completion of H such that 
IS invertible (Suen conpletions are treated in greater 
d tail in Chapter 6) 
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L6t d 1 1 2 n be the diagonal eleTnents of 

(H^ H )'^ In the following theorem prov nat 

^gx ^ ^Lx (5 34) 

11 is ©asy io see that (5 33) follows as a special case of 
(5 34) when |kl = 1 


iheorem 5 3 Given dj^^ as defined above 

1 2 V j\2. ^ . 

‘‘gi '^Li 1 = 1^ m 

*T' 1 

Proof (H^ written as 

g y 


‘«g «g> ' 


T T 

H H 


He H He 


‘11 

*^12 

'21 

*^2 


oince 



We have 

^11 ^^^c^21 “ -‘•m 

^12 ’’’ ^ ^c^22 “ ^mxn-m 

Therefore from (5 *^6) w obtain 
hTh^ = 


(5 35) 
(5 36) 
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— 1 T 

v;her^ exists becau e F H is a posiiive definite matrix 

^ 2 . 0 a 


T 

substituting for H rl in ( 35) w get 

V/ 


g g 




^11 '' ^ 12 ^ 22^21 


(5 37) 


u 


21 


r 


a H F IS a symn trie matrix 
iz 9 y 


-1 

No\; let be the diagonal ^1 ments or ^^2^22^21 


Since = G ^2 ^22 ^ P ^ 


^12^22^21 ^ ^ definition oi a p d matrix A 


X > 0 for all 5 e s” xcept x = o 


which snows that 6"^ is strictly positive Now using (5 37) 


d? =d2 

Li gi 




v^here 


gi 


dZ. are the diagonal entries of Gj^j^ 




oummarizing in this chapter we have s udied the applica- 
tion of k Circulant completions for data reco / r/ in the con- 
text of BDT tie hdve analysed two DR schemes v^hich facilitate 
good reduction in omputational complex t/ and torage require- 
nents compared to the DR schemes studied in the previous 
chapter Horeovor these schemes are am nable to oerformance 
analysis U have obtainec explicit expressions for the proba- 
bility of rror and cornpar d it with th p rfornanc achievabl by 
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DR scheme based on LS-decor ^;olutlon In general the perfor- 
mance of DR schemes based on k Circulant completions is infe- 
rior to the DR schemes of chapters 3 and 4 However in 
Chapter 8 we shall see that in the case of some channels 
DR schemes based on k-Circulant conpl tion perform well 
compared to ML-DR and DF-uP schemes 

In the next chapter w take up a decaileo study of the 
relationship betv/een least squares deconvolution and deconvo- 
lution via k-Circulant conpletions 
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CHAPTEt* 6 

L ST SQUARES DCCOIvIVOLUTION VIA k CIPCULAIlT COIlPLETIOiMa 

recall fiom our discussion in the previou chapter 
that a olution o xhe decor volution problem via nonsingular 
k~Circulant completion can b express d as 


-k ~ ^k2 ^ 


where = [H = 


k2 

k3 


(6 1 ) 

This approach is 


computationally attractive compared to the least squares (LS) 
solution given by 


Xls = (H^H) ^ r 


(6 2 ) 


.-1 


becaus can be obtained using FFT However in general 

— Lo ^ seen from the following simole example 




1 

0' 



" 1 2 ■ 

Example 6 1 

Let H = 

1 

0 

1 

1 , 

and 

r = 

IKiM 

0 5 

-1 


Foi k = 1 


^1 = 


10 1 
110 
Oil 


o 1 - 1 
^1 " 2 


1 1 
i-1 1 

1 -1 


Hi 2 = t 


11-1 

-111 
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But = 3 




1 -1 

1 2 


Ther fore 


X 


LS 


1 lo 

-1 23 



X 


1 


Thus it becomes impo tant and interesting to know condi- 
tions under wh cl “ Ic ^ caus in su r a case LS solu- 

tion can be obtaine 1 van nuch less cornpuxation via k Circulant 
completions 

In the present chafx r we study chis problem in a more 
g n ral framework of square completion of rectangular matrices 
Let Hg = [H be nonsingular square completion of H 

in gei lal need not depend on H Tie k Circulant comole- 
tion of H IS a oarticular case of wh re depends on H 

Our interest is to obtain conditions under- 

which 


Let = 


X 


LS ^ 


X 

s 



(6 3) 


how that x, = x if and only if - O Such 

SI rnxn""rn 

completions are called or hogonal co ipletions As a corro- 
llary it follows that iff 

this condition we fur h r show that orthogonal k-Circulant 
completions of H are orbhogonal k Circul nt matrices 
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Wp organize the chapter in the folloving manner In 
Section 61a method of solving overdeterr in d set of linear 
equations using the notion of squar conpletior of natricos is 
discussed A '^riof review of least squares solution of lin ar 
system of equations and Moor '■Penrose (/IP) inverse of matric s 
IS given in Section 6 2 In oection 6 3 jc obtain condi + ions 
on squar completions that result in P-inv rse or least 
squares inverse (LS-inverse) of a full ank rectangular matrix 
Sauar completions which result in /nP-in/erse are called ortho 
gonal square conpletions V further sho j that is an ortho- 
goml k Circulant completion of H iff is an orthogonal k~ 
Circulant In Section 6 4 we determine channels which are 
amenable to LS solution via k-Circulant completions \/e show 
that in most of the practir 1 cases LS solution to the deconvo- 
lution problem cannot be obta ned via (62) because in such 
cases k-Circulant completions do not result in orthogonal k- 
Circulants In Section 6 j v^e charactenz orxhogonal k- 
Circulants in the k-DFT oomain An altcrnat expre sion for 
LS solution using partitions of k~Circulant comoletions of H 
IS obtained in Section 6 6 for the purpo e of knowing the 
relaxionship between Hj ^2 >iP-invor of H /o show that 

h" = «k2 - Hk2 <»k3 ^3 


wher H is the Moore-PenTos inverse of H 
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6 1 SQJoRfc COnPLnTION Ai^ID SOLUTION OF LIM EQUATIO J 

s the doconvolution problem is assent ally solving a s t 
of linear equations we first consioei aiplicotion of square 
comol tions for solvino a system of linear equa+ ons giv'^n by 

AX = b (6 5) 

where A is an nxm matrix with n > m Let i nk of A(,r( \) ) be m 
Let Ag = [a Ag 2 ^] be a nonsingular souare completion of A 
Such a choice of A^j^ is possibl b cause ^ is a full rank 
ma rix Lot x be an ext nded vector 

mr- 

y = (6 6 ) 

U 

ifh r o IS an n-m length zc o v ctor In t rms of we can 

o 

ob am n solution to (o 5) given by 


X 


e 



(6 7) 


Koto that 5 X xn g ncral lie will return this ooint 
later Now let 






(6 8 ) 


whoT-e IS a solution o (6 nth rospcct to h>- comole- 
t on Agj^ Depending on th c oic of A aifr lent solutions 
arc obtained natuial cuesx on th d -nands answer at this 
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stage is jheth r all such solu 3 ons ar equivcjlent in so e 
sense or nox Because if th solution given by (6 6) i 
ind p ndent of it niy oe economical in terns of computa- 
tions to choos sucn tha inversion of in/olves least 

SI s 

ornputationd complexi y One si pie resul in this connect on 
IS th folloving 

Theorem 5 1 If b bolorgs to th range space of then 'x 
giv n by (6 7) is indeoend nc of 

Proof L t -ni-i columns of A Since b 

is in the range space of A b = 

some s Therefore solution to 1,6 ) is given by 

^ n = £ 


L t A be any nonsingular square conplotion of / 

A, 


Let 


-1 

s 




's3 


Then from (6 8) we see that 
b = 


X = 


s2 




Since we obxain Ag 2 A = If^ fherefore = £ 

which is indeoendent of \ „ 

Note that the condition on b demanded by Theorem 5 1 
cannot be guaranteed in practic i b does not belong to 

the range space of \ ie consider this case in S ction o 3 
In th next section we give a brief reviev/ of the conventional 
methods available for d aling with this case 
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6 2 lEPST square ^AI 111 Ui OLUTIOr lU f GORE P^mOSE 

G^vlEPiLIZED IinIVERSE 

ihe rretJ'o^ of solution normally used v/? ep b is not in the 
ra je sp ce of a is to constrain the solution space The 
m thod of 1 ast quares is one way of constraning the solution 
p ce /e choose only those y s v/hich minimizo 


I ^1 = ( X ~ t>) 


(6 9) 


when A is of rank m it can be shown that the right hand side 
of ( 69 ) IS minimized by 

A^b 

when does not hav full rank (t(Aj < the method used is 
to further constrain the solution space b/ considering x s 
hich minimi z (6 9) and have minimum length It can be shoyvn 
the such a solution is unique and is given bv [85] [87] 

-mLS 

where ^ is called the \Aoore-Pc arose (IIP) invers of ^[85] 
s tisfies the following properti s 


1 

2 

3 


A+AA”^ 
VA •") 


= ^ 




= AA 


(6 10 ) 


4 
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LS LS 

A matrix A which satisfies the conoitions fi = A and 

LS LS 

(AA ) = AA is called a least squ re inve se of [0 J 

LS + 

When \ IS of full rank it can be shov/n that A = A 
3 L /oT SQUr^PES SOLUTIO'J \/I bQU/R CO IPLETIU S 


Another approach that can be taken when b is not in the 
range space of is to e tend the solution space It is easy 
to s e that squ i completion is a w y of extending the solu- 
tion space The solution obtained by forming n-^b depends on 

o 

;hich controls the solution space In such situations it 
is in t uctive to know \ lat kind oi corioletions lead to I'^a t 
squares olution Alt natixely we are int re ted in s 


which ma! o ^^2 ^3 


-1 


2 

s3 


a lea t squares inver or 


loore-Ponrose inverse Th following lemma characterizes MP 
inverse of A via square completions 

^s2 

=1 / 


s3 


Lemma 6 1 Let A^ = [A and A^ 

Jt- ' ^ 

IS the MP inverse of a i-^f (Agj|^Ag 3 } = gi^s 3 
Proof h know that A~ '"A^ = = I_ Therefor 


Then A 


s2 


h know that A”^A„ = ^_A"I^ = I 
s s s s n 


in terms 


-1 


of the partitions of A and a th following matrix equations 

S 3 

must be satisfied 


s2 ~ 

s2 sl~ ^mxn-n 

^ "0 
s3 n-mxm 

s ^sl“ 

^■^s2 


(6 lla) 
(6 lib) 
(6 11c) 
(6 lid) 
(6 lie) 
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Jsing (6 ila) it is easy to shov; t satis los the first 
three properties of he f P-inverse of a given b/ (6 10) The 
ou-^th proo rty in (6 10) requires t lat 

^ ^ ^ s2 

From (6 lie) we ca » so that this condition will b satisfied 
^ si s3^ "" si 3 

Using L mna 6 1 we can prove the follov/ing tneorei 

Th orem 6 2 Gi yon a nonsingular comol loi of ^ 

/^s = [ ^g]^J and it inverse s ~ ^ s2 ^ ‘ ^ 

invers of iff = Onxn-m ^ ' 

Pioof Let Ag 2 1 1 P inv rse of \ Then ^2 ~ (A^A)”^A^ 

T 

ubstitutirg for ^ 1^2 m (c 11b) wc see that ^ 

This Droves the first part of the theorem 

Now assume that A‘A„, = O Let (A) denote the 

O Jia iTlydl***r* 

r nge pace of Then R( A) _L R(Agj|^) Here X indicates that 

k(A^i) is orthogonal to R(a) From (6 lib) we have 

f{ A^ 2 ) -L ^(Agj^) Since P(Agj^)x T(A) it follovs that the rows 

of A„ , pans R(A) Bub columns of -v also sp n P(A) So 

T 

" DA tor some nonsinjular matrix D Subs t tuting for A 2 
in (6 IJ n) we get D = (A^a)""^ fha irplies A ^2 ~ ( a'^A) ^ ^ 

From now onwara we shall r for to square completions 
having th property A A 1 = „ as orthogonal completions 
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In th nex-f th orem j consider the impli otior o Theorem 6 1 
in the ca e of l<~Circulant completions 


eoren 6 3 C.iv n a nonsinqular k~Circulant coipletion of H 

kl JL hLi 1 V t? L rij^ 1 ^ 


h = [H and its inverse = 


\2 


k3 


i j 2 IS the \P- 


in^ rse of H iff i an oithogonal k-Circulant i atrix 


Proof From tho oefinition o k-Circulant latrix (Appendix B) 
V c 1 1 that h^ th irst colinn of Hj is related to oth r 
columns ox through /;hich is a k CiTculant matrix with 
first ro/ (00 k) n othcrwords 

hi = 

wh rc h^ IS th ith colun a of Fj^ 

Using Theor m t> 2 know that Hj ^2 ^ inverse of H 

if I = 0 

kl nxn~m 

For this condition xo hold ve must hav 

h^h^ = 0 j=12 m 1 = 11+1 n 

(6 13) 

In view of (6 12) (o 13) molies 

i?! = 0 3=12 m 

1 = m+1 n 
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T 2 

But \ppenoi B) 

Ih^^refore we obtain = O 

o- ~ ^ 1=12 n~l 

Bu "O^hj^ IS the ixith coiunn of Hj^ 

Iherefor in this case is an orthogonal ic-Circulant 

* 

Fron the forogo ng di cu sion it is clear that the 
square completion of rectangular matric s provide a general 
framework for the stuJy o deconvolution problems Both 
the 1 ast squares approach anc th m thod based on k~Circulant 
complet on are particular ca cs with appropriate choice o the 
completions In the light of Theorem 6 J in the next section 
m d termne those channels which admit LS solution via k~ 
Circulant completions 

6 4 CH/-JVJNELS M-lt lABLE TO Lo SOLUTION VI k CIRCULANT 
COMPLETIONS 

In this section we are interest d in determining those 
channel convolution matrices 4 such that is an orthogonal 
k-Circulant Two cas s are to be considored 

Case 1 m > g 

In the case of block data transmission m > g This is 
n cessary oecause otherwise xeduction in th erfective data 
rate o/cr th channel will be 1 rg Thu form of in this 
case is shown below 
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h. 


0 

h 

h, 


o 


0 


0 0 


0 


h. 


kh 

c 

0 

0 

hi 


^g~i 


kh 


0 1 


kh 


g 


u 


h 


g 2 


kh. 


kh^ 


kh 


'’l ho 


I IS eas/ to see that for Hj^ to be n orthogonal k-Circulant 
the following equations are to be sati fied 


kh^h„ = 0 

0 g 




= 0 


k(hohi+h^h2+ 


Since ITq ^0 we obtain 


^ ^g-1 




= 0 


= 0 


This means that the only channel that admits LS solution via 
k~Circulant completion is the distortionless channel having 
h^ 0 and all oth-^r ISI components zero 
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/ now consid x the case n > m+g This case axis s in 

most of the deconvolution applications For instance vhen 
TFT IS to be used for inverting H It is oesirable to hav n 
a poA/er of 2 in such situations Therefore wh r n is not 
a power of 4. the channel vec l-or is pa ded with zeros Another 
motivation for consid ring this case is that ere nay exist 
some n > m+g such th t LS solution is obtained via £<-Circulant 
completion 


In this case H. 3 11 ha /e a more general form as shown 


b low 


»ke = 


H H 


VA 


H 


ko 


v/here 0 is a null matrix of order n m-gxm Using the arguments 
given earlier we can easily show that is an orthogonal k 
Circular t only when h^ 7*^ 0 and h^ = 0i=12 g 


Case 2 m < g 


This situation arises in convolution si uations where 
the number of unknowns is less than the number of impulse res- 
ponse coefficients 

In this case takes the folioring form when g = 4 and 


m = 2 
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' ^ ^ 

kh4 

khg 

kh2 

khf 





>^1 ^ 

0 

kh 

kh3 

kh2 





h2 hj^ 

ho 

0 

kh^ 

kha 












K 


hs h^ 

h 

0 

ho 

0 






^4 ^3 

^2 


ho 

1 

0 





i 0 h^ 

hs 

^2 

hi 

h i 

o 



For 

Hi 

to be an orthogonal 

k-Circul ant nus+ have 


kh^ 


+ h3 

^ - 

hh 

= 0 




‘‘he 

1^3 

+ h3 

h + 
o 


= 0 





hi + khj^h2 

+kh2 

h 

h3h4 

= C 




Any solution 

to 

the 

DOVC 

set of ecuations s 

rves as 

th« 

de 

ired channel 

Thus 

for 

■ m < ' 

g we e thit 

it ma / be 


possible to obtain LS solution via k-Ci'^ ul nt coTipletion 
Explicit form of s is difficult o obt in n Lhis case 
in the n xt section we give n alt rnative characterization 
of orthogonal k Circulani /hich helps ii obtaining some 
insigit into th natu o ox the possiole solutions 

6 5 CHAPACTERIZATlON OF ORTHOGOnAL k CIRCUL^NTS 

From Appendix o we have 

Hk = ^k^ di g ( Xq Fj^ = A 


(6 14) 
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and = FD v;here D = diag(l p 
Using these in 



(6 15 ^ 


(6 16) 

(6 17) 
(6 18) 


The oroblem of determining orthogonal k Circulant thus reouces 
to he detern^nation of A such that (6 18) is sa+isfi d for 
the given k I''" none of th elements of / are zero uue to the 
diagonal n-^ture of A it is asy o see that s iiust satisfy 
th condi ion 

[Xq 

= C (matrix of all ones) 

(6 19) 




To see this n = 2 Th-^n will be of the form 


X = 


ihen l\ -^ (\ = 


X 


X 


11 

12 

21 

X 22 

^11 

X^X 

^ X 
1^21 

2 ^ 


In order to satisfy (6 38) ve must have 


= 1 ^2 ~ ^ ^2 ~ ^ 


The procedure for obtaining an ortho jon 1 k-Circulant is as 
follows lor the given v lue of V (6 18) rru t be solveo to 
obtain s which in turn proouce via ^6 14) ks an 
llustration we con icier he case when I = 1 

\Jhen k = 1 (DD’^)"'^ becomes an identity matrix Hence 


X = 


1 

0 

0 

0 


O 

O 

0 

1 


0 

0 

0 

0 


0 

1 

1 o 
o 


U ing (6 18) wo s e tha ^ i ® hogonal circulant 

satisfy 


2 
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For a real matrix w have 

\ =^*- 1+2 ^=2 3 n with 

Therefore 

1=12 \\ where \ = 

Different choices for ^ ^ 2 ^ i decioe different orthogonal 

real circulants Thus vje have infinit numb r of orthogonal 
Circulants The s are given by 

A ^ = cos0^ + j sin0^ O i. 0^ <2% 

For instance when n = 4 we can have 

A^=l ^2=1 JsinQ 

Then the first row of the desired k~Circulant is given by 

h^ = (l+cosO sinG l~cos0 sin9) 

6 6 LS SOLUTION USING k-CIRCULANT COMPLETION 

From the analysis in the previous sections ve conclude 
that for most cases of practical interest LS solution to the 
decon solution problem cannot be obtained via k-Circulant comple- 
tions as given by (6 1) Hovever although he solution given 
by (6 1) is not the LS solution it may be very lose to the LS 
solution In this secxion v;p obtain an alternate expression for 
the LS solution using k-Circulant completions 


+ 1 it n IS even 


n+1 

o 


It n IS odd 
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Consider the received vector r given b/ 

r = Hx + V/ (6 20) 


We first obtain the LS solution using heuristic arguments If 
/j w re to be a zero vector and exists for some k we can 
obtain exact solution to (6 20) as 


X 


Xi = H,^2 ^2 


^k3 


r - 0 


(6 21 ) 


wher 

= [H Hi^i] 



On the otherhand v;hen w is not a zero vector we have in 
general x 7^ x^ and Xg ^ ° ^ ^2 

2.2 = Hk3 i = H^ 3 (Hx + w) But H^jH = Therefore 

Hk3 £ = % 22) 

The n nimun no m 1 ast square solution (6 22) is given by 

« = H^ 3 (H 1 c 3 H; 3 )"^ H |^3 r (6 23) 

Thus we have estimated the noise vector w in the best possible 
manner under th given conditions ie can use this estimate 
to reduce the ^ feet of noise in (6 20) Substituting for r 
using (6 20) in (6 2l) for Xj^ we obtain 


- Hk2 I = + Hk2 


Using (6 Ao) for w we get 


H |,2 r = X + H ^3 r 

or X = H^ 2 [i - 4.(Hi.3 

/c novf claim that no b ttcr result can be achieved under th 
given conditions A forrral proof of lus claim is given 
b low 

Tl orem 6 4 = H^ 2 [l-H, 3 (H^ h;J ^ k.j] (6 24) 

Proof ie show that (6 24) sat sfies 11 the four propertie 
of tne VlP-inverse given by (6 10'' Using (6 11) whe e ins ead 
of we have and replace a“^ vath 

^k2 ~ ^k3 ” ^s3 et,sy to show that -representation 

of F given by (6 24) satisfies th following* condit ons 


HH'^H = H 

and (H'^H) ^ = h‘*'H 


'/e hav to finally shov) that (HH"*") ~ HH"*" For this we make 

use of (6 llG)which gives after appropriate subst tutions 


««k2 ^ «kl\3 - 


Therefore 


k3^"k3'‘k3' 


‘k3' 


= tl-«k3 (HksHks) 


1 9 

k3^ ~ ■'l I'^kS ^ l^k3 
= [I-Hro (Hk3 Hk3>"^ f'w] 

vjhich IS a syt If trie matri 
i he re fore (HH^) = 

But such a matrix is uni jue [35] 

-St'^ 

In Lhc forcooing oiscu ion jg u ad heuristic arguments 
for obtaining (6 24) ^ov/ we will show that (6 24) can b 
derived in a more for la^ manner Fo"' this purpose we write 
{i' 20) usinc k-Ci culant completion of H and extended vector x^ 
as 

£ = Xg w (o 23) 

Jote that X is an e^^tended version of x oot m d by 
•*-0 

padding x with zeros However Afhile s eking solution to 
(6 5) via we ignoreo this constraint on x^ Th rofore 

the solution naturally differs fro a Xj^g xceot in some 
special cases This means that in oroer to obtain LS solution 
to (6 25) we must minimize 

(r - Xg) (r - H|^ x^) (6 26) 

subject to the constraint 

C X = 0 (6 27) 

G 

^ ~ ^^n-mxn-g 


wh "e 
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This nininization proDlem can be solved usin^ Lc^range 
rnul ipli r m hod Lot a = {<y^ bo the mtitipliers 

Tnen c fo m 


r(x a) = x^) (r~Hj^ x^) + 2r ^ C 


X 


Different ating F(x a) / th rospoct tu x and a v/o obtain 
conditions for mini lum as 




”k Hk io £ r CC = 0 


and 


C x„ = 0 
e 


AS IS invcrtiblo we obtain 


and 


-1 ~1 T 

^0 = Hr £ - Hk 0 

a = (c H*; ^ c’')"^ C h;;^ 

-1 Tv~l 


existence of C ) follows by noting that 

H., 


- Hj ^3 where =» 


and Hj^3 is a full rank matrix Thus 


'k2 


‘k3 


ie = Hk^ r - H*" H^3 x 

Therefore the solution to tne original problem is given by 


2. - ^k3 ^ 
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U ing Th or m 6 4 we sgq that x is the LS solution bo th 
deconvolution problem 

Using (6 24) ve obtain an xpression for the di-^ er nc 
AH = - H given by 

4H = Hk3 'Hk3 “ks’*^ 28) 

Note that Ah 3s ind pendent of Therefore it c n be 

computed b> kno/ing 

If the 1 ments o H ar<^ small the solutions obtained 
using hj ^2 will noc uiffcr s giificantly 1 1 th case 

of orthogonal k~Circu] nt completions of H it is easy to see 
that ah = as I = ^mxn-m 

In summary wf* hav stuoi d LS U c nvolution problem in 
the framov/ork of squar conpletion of rectangular matrices 
We have shovm that in lost of the oractic 1 situations LS 
solution cannot be obtainea via dir ct inv rsion of k 
Circalant comoletion of the chann 1 convolution natrix 

Another method of ootaining LS solution is by using 
It xative schemes In the next chapter we study steepest 
descent algoiithn in th discrete Fourier transform domain 
which provides LS solution Lo the dcconiolu ion problem 
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CHAPTER 7 

L ^ST SqUAR^o DhCONVOLUTIOM - I fcRATIV^ APPROACH 

In thp previou chapter we consiuered th pos ibilit/ of 
applying computat onally attractive k’-Circulant comoletions of 
ch nnel matri H for obtaining L solution to the de onvo*- 
lu ion p oolern which involves inversion of H H Our analvsis 
sho /ed that in most of the practical ases the solution given 
by ^6 l) using 1-Circulant comoletions di not yield the IS 
solution to the decon /olution problem -jn alternate expression 
for L solution v/as also obca-ne l in S ction 6 6 Though th 
de"'ivation of this expression inak s use of k~Circul?nt comple- 
tions the final expr ssion is not compute lonally att“'active 
compared to rhe meet in/ersion of H^h 

In this chapter we study ano her metho i of ootaining LS 
solution to the oeconvolution problem It is based on the 
application of weiiknown steepest descent algorithm (SDA) [89] 
which IS an iterative technique in contrast to the noniterative 
or cirect approach consideied in Chapter 5 

Gersho [24] appears to have used ixed step SDA for the 
first tim for the purpos of adaptive ch ai 1*^1 eou lization in 
which the tap oefficients o^ the equalizer are adgusted conti- 
nuously so that the mean square error between the actual and 
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s + imated data synbols is minirnized He has shovm thnt in the 
or sence of noise the algorithm yields a suboptin 1 setting of 
the tap coeffici nts 

\ialzman ana Schvjartz [2l] [2_] have stuaied fixed step 
oU/j in the DFT dom in for the ou pose of a tom ti channel 
eoualization Usii two examoies they ha/e obser/ed ti at th 
DFT domain SD^ conv rge faster than the sample domain SDA as 
exact I nowledg of t^ eigonvalu are avail ole in th case of 

DFi ooraain SDA Ti^ir work has he n extended by Picchi and 
Prati who hav inciud d self orthogonal zing feature into 

the JFT doi 3n i ration scheme Using simulation studies th v 
have sho//n tha the per ornance o self~orthogonalizing DFT 
domain quail zation scheme is sup r or to that of the sample 
domain algoriuhm cu to Godard [dSJ 

The observations mad oy /alznan and Schv/artz [21] and 
Picchi and Prati [64] ar more carefully ex -ninea in this 
chapter One re son for this is based on o ir und rstanding 
that DFT b ing an orthonormal transfornacioi th it ra ive 
sch res in both tha domains must have same convergence proper- 
ties But th-^re i ay be co iputatj onal advantages in going rom 
sanplo domain to th DFT a rain du'^* to th availability of fast 
algorithms for computing DFT 

l/e devot a lajor portion of this chapter towards a b tt<,r 
understanding of tho “'olo of DFT and k-Circuiant completions in 
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reducing processing tin ano orovi^ing last converg nc Our 
naly is clearly shows how DFT helps in redaciny conputa ions 
Jsiny rh fr me ;orI o l~Circulart comolctions u show that 
dvant go in t T’ or f ster converg nc r t can be chi v q 
in the DFT domain b caj it p o/ides n th c sc o some 
xanples Dood stinraio of the con /erg ic ara r ter it i 
horn th t th cstnate obtained using k-Circulart c nolction 
can also be used in tho sample uon in to obtain s me conv r- 
g nc ra 0 

"^h Chao ter is orgcOiz U in the fol lowing rnann r A on f 
r VI si of steepest d cent aljorithii is given in oection 7 1 
Ihc n nne n v/hich DFf helps in r ducino computat ons is dis- 
cuss d in S ction 7 2 In oection 7 3 we oori /«■ SuA in th 
DFT domain from thro c^. f i nt vi v/points oat recover/ 
s Che no basoo on DFT dome n bDi is iv n in oc ction 7 3 1 
Converg nee oropertics of SDA r discuss d in Sc tion 7 4 
method of increasing the onverct nee rate of the SD is dis 
cus d n S<=>ction 7 5 te show ho / Girculant or k-Circulan 
comoip ions c n be uscct to r due nunbei or itor tions This 
analysi orovides xpicnition for the f st convergence beha- 
vior o he self-orthogcnalizinc DFi cJonaan scheme srudied by 
Pxcchi ana P-ati [64] In cti 7 6 v/c stov; ho stimat s 
of step length in the cis f iix a t«p uDA can be obtained 
using eigonv lues of th k-Circulan completions This explains 
the observation mad by i l^nan nd Schvarez [21] recording 
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fe t conv rgenco of DFT domain schen-^ conpd eel to sample domain 
sc r e 

1 OF STuLPEoT DE'dCi-i\'" LGORITFil 

in this s ction w oiscuss briefly c rxain g neral f atures 
of he /jellknown Cau hy s steep st d see it Jgorithn [89] Con- 
sid r th qu drati obj ctiv f notion qiv n by (2 8) 

F(x) = x^h'^^Hx ^ 2x^1 - r (7 1) 

The SD constructs & sequenc of v ctors starting \ ith 
chos n a bitraril/ given by 

;h r IS dot rmined such that i-S ninimizcd Substitu- 

ting (7 2) in (7 1) and diif rentiating r to it can bo 
shov,/n that is given o/ 

T 1 

a = e, = H'^(r - HxJ (7 3) 

Comoutation of at ev ry iteration 3s the most time con 
sun ng part of SD/ Thcr foro in nost of th practical applica- 
tion 1 fix d at a value a chosen such th t th optimum 

speed of c nverg ncQ is achieved It can b shown [39] that the 
iteration scheme (7 2) conv rgos to th LS solution 

x^g = r 
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I may be int rest nj to note It t the fixeo step SDA 
(a = O' ^ all i) is essentially a j neralized version oi the 
Jacobi iteration [74] for solvina the linear e nation 

J^H/ = 

T iteration scheme in this cas ta es he fo r 

-1+1 ” 1- aH^r (7 4; 


This iteration scheme can oe shovn to converge f 

oo 1 lum value o a is jiv^n by 


a 



2 

lax 'run 


(7 5' 


where and ar«> the maximum and ninimum eigenvalues 

T 

of 1 H respectively [90] (see page 24) 

7 2 APPLICATIOI] OF DFT TO STEEPEST DESCE T ALQOPITHiv^ 

V now consider the possibility of using DrT to reduce 
computational load associated with SD4 Consider e^ given by 

= H'^(r - Hx^) 


Let be the Circul nt comoletion of H and be the extenaed 

vector formed by padding x^ with n~m zero Then w can write 

(7 6) 


= hJ (r 


Hi x,„) 
1 —le' 


where 




'll 


and 


il “ ^ll^-~^l^ie^ ^1 ~ 
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From (7 6) it is clear that if we compute the ri ht hand 
s de of (7 6) is obtained as he first n ntries of 

]ow let deno^ the Dt*T of In ern of the DFT of r x 

and h the first coluiin of we can v;rite (7 o) in th DbT 
uo nain as 

= FH^r (Fr - FH^F Fx^ 


= F. 


( i H, 


le' 


(7 7) 


r fore by knovanq P anc ^ the r h hand side of 

(7 7) IS easily comouteJ and is then obt ined as 

= PF'*^ fc (7 8) 


wher P I 

Now if E denotes th DFT of £ where e is an exten~ 
ded vector obtained by padding with (n-n") z ros given by 

(7 3) can be written as 


a 

1 


e^e / 

— 1 i' 



H, 



(7 9) 


fh IS by knowing the Dbf of e^^ computation of becomes 
relativel/ simolo task as v^e already have the knowledge of 


It shoulo bo noted that the value of remains same both 
in the sample domain ana th DFl dor am Accordingly the 
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converoence rate renains unaltered b/ the application of DFT 
V/e will com back to this point later 

Further we wish to note that ns k~DFT computation using 
rri algorithm requires more conputation onDarad to DFT k- 
Ci" ulant completions othfi than k = 1 arc not as useful in 
rrducing computations per it ration conpar d to Circulant 
completion 

7 3 DbKiV TIOu OF THc DFT DO lAlN STECPbST DloCENT ALGORITHM 


In this section /o derive oDA in the DFT domain from thr e 
diff rent viewpoints The obj ct vo is to cl arly bring out 
th Tolc of DFT in Sua 


Approach 1 Using the extended vector x and Ci culant comple- 
tion of H v/o can write (7 2) ai ornativel/ as 


i(i+l)e = “i ’’ Hi 


where 


P == 


n-nxn 


(7 10) 


P reraultiplying tho above equation bv th DFT matrix we 
obtain 

K(i+l)e = ^xo “xFP 

wheie / - Fx Since F^F = we can modify the above 

le le n 


expression as 
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itow and ’^H^F are diagonal m trices vith U^T of th 

first column of and its complex conju ate a d3 agonal 
elements respectively Th refore we can vjrit (7 11) s 

^(i+l)e = ''xe F l-I (P-Hi>:xe) 

Moreover we can write (7 12) also as 


'(i+l)e ""le "** 1 ”1 


where 


, = FP F ^ 

-1 1 


= h, (_P - Hi 43 ) 


can oe expr ssed as 

T T 


a. 


-le -1 ^le^ 


(7 13) 


(7 14; 


(7 15) 


The pair (7 13) and (7 lo) repre ents D T domain 1 ration 
schrm s corresponding to (7 2^ anc (7 4) respectively 

approach 2 In Approach 1 v/ derivea DFT domain SDA starting 
from sample domain oDA In the pre ent approach vje formulate 
the decoivolution o oblem as 
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min (r 
z e 


(r - Hz) 


subject to Cz = 0 


(7 16) 


where C = (7 17) 

The minimization problem given by (7 16) involves synme- 
xric Circulant positive semidefinite matrix in general It 
mav noted that even though is oositive definite 

need not alwa/s be so \ sirpl il-.ustrativ xamole is giv n 
belo\i 


Consider 



1 

0 ' 






T 

2 1 

H ~ 

-1 

1 

cl-^arly Hr = 

-1 2 


0 

-1 


- 


IS positive definite 


wh rtas the Circulant comoletion of H 



1 

0 

-*1 


2 

-1 

-1 ' 

Hi V 


1 

0 

T 

gives H^hj, = 


2 

-1 ■ 


0 

-1 

1 - 


-1 

1 

2 


T 

which has 0 as an eigenvalue Consequently is positive 

semidefinite 


It may be furth r noted that whereas nini ization of F(x) 
gi /en by (7 l) is an uncons trained probien (7 16) defines a 
constrained minimiza ion p oblom Sich problems can be solved 
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using Rosen approach of projecting the graoi nt at every 
ittration on to the null space of C [89 j [91] L t G b a 

natrix such that CG = ti \,7 17) i is easy to 

tl at f' can have the following form 


G 


I 

O 


m 

n-m m 


Tollowing tic approach given in [89] (see pages 260 266; 
w can write aown the iteration scheme for (7 16) as 


z 


14-1 ■^1 




(7 18) 


T 

wher a, = min F(z ,,) but QG = P i/e can now use DFT 
1 ^ - 1+1 

dn>-. write (7 18) as done in th cas of approach 1 in the 
DFT domain Thus Afe obtain 


L 


1 1 


z, + ^FP r*(H‘ r - h,^ 


{^ 19 ) 


Since plays the same role as in 13) we s that 
(7 13) and (7 18) aie identical 

Approach 3 W now consider the approach taken by Valzman and 

Schvjartz [2l] and Picchi ana Prati [b4] for deriving DFT domain 

SDA Using F and the Circ ilant cjmpl tion of 1 we can rewrite 

th minimization problem given by (7 16) as 

T 

mm (Fr FF^(Fr FH^i) 



1^)2 


writing FHj^^ 




and 


noting that Ft- 


min (P >, Y)* 
Y 


(R-rij_ Y) 


= \;e obtain 

n 

(7 20) 


v;h re by our earlier convention r = Fr i = Fy 

No in terms of Y e 0*^ we can foi^iulat the mini iization pioblem 
given by (7 16) o uivalently as 


T 

min (P-Hj^ Y)* (R -i--) <) 

yed^ 


(7 21) 


subject to 1) F Y 1 real vector 


ana 2) CF / = 0 


wh re C is same as in (7 16) ^^olution to he above problem 
car be obtained using Hosen s gradient projec ion method as 
de cribed in [21] Alternatively since (7 21) is d rived 
using (7 16) vhich has (7 19) as the corresoondii g SOA in the 
OFT domain it follo/js that (7 21) s'* so v/ill have (7 19) as 
Its corresponding oDrv in the DFT domain 

At this stage we vash to note the follow nr distinctive 
feature of the three approaches rtoproach 1 c eals with a 
unconstrained minimization proileii In Approach 2 we deal with 
a constrained minimization problem \ath one constraint In the 
third approach Ne have to deal with tvo constrainrs The form 
of SO/ in the DtT domain hov/ever remains unchangea Likewise 
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Ve convergence oaranieter also re lain s unchang d by he 
apolication of DFT Thus we can conclude ha DFT and th 
associated Circulant co ipl tion I elp only in reducing the 
comoutational comol xity of th iterativ scheme These 
asoects donot b cone clear ix the SDA i d rived using 

Aporoach 3 alone because solution of the r inimization probl n 

given by (7 21) do s not sh w explicitly its relationship 
with oDA in th ample lomain 

731 Data R covery Schen based on SDa in th DFT Domain 

A DR sche le baseo on DFT domain iter tion sch m (7 13) 

IS sch natically sho ;n in ’"igu"© 7 1 In this scheme the 

staiting ooint s chosen as 

^oe = BlUl 

I'lote that in the absence of noise and when the Cir ulant 
completion of 3s nonsingular repres rts DFT of the 

actual Jata vector padded with n-m zeros Therefor'^ in such 
situations the iteration scheme w 11 converge in one iteration 
On the otherhand when noise is present provides a good 
starting point oc he iteration s hem The p rfor iance of 
this scheme will o sane as that of DR sch ne based on LS 


solution 


FFT 
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I 


I 



FIG 7 1 Data recovery scheme based on DFT 
domain steepest descent algorithm 
















7 4 RAlE of CONV HGENCE OF SDA 


oDA gener tes an infinite sequence of s whose limi is 
the solution to the optimzation problen Due to vere con- 
straints on processing tim in a comnuni cation situation large 
number of iteration will not be a desirable feature Therefore 
1 b come important to know the speed of convergence of SDA 
In this connection we have the following tneorem [89] 

Theorem 7 1 Suppose that x is the solution to minimization 
of FCjc") given by (7 l) L t c^ be the condition number of 
That IS Then (see [89] pag 146) 

F(=£n.l) - F(^) < (^r)^ [F(Xl) - F(x)] 

From Theorem 7 1 it is clear that SDA has line r rate of 
conv rgence The rate of convergence depends on th condition 
number of the channel autocorrelation matrix If = 1 the 
iteration scheme onverges in one iteration But as in- 
cr ases tne convergence rate decreases and hence large number 
jf iterations will bo needed oefor mcaningiul estimate of the 
solution vector is obtained Thus SD is not an efficient 
method when the condi+ion number of tic c lannel is large One 
mothoa ox ccv„lerating the convergence rat of SDA is by 


altering the eigenvalues of the channel autocorrelation matrix 
so that condition number is near to unity In tn ncx section 
V consider this aspect 

7 b FAST CONVERGING STcEPl^ST DESCcNT ALrORITHM 

Consid r th it ration scheme given by (7 2; The schc ic 
conv igcs to the exact solution when 

H^(r ~ =0 (7 22) 

for some i Thi means hat in tcaa or wo can substitute 
aq wher Q is a nonsingular matrix and obtain another- itera- 
tion scheme 

•^1+1 = ^1 aqH^^r - Hx^) (7 23 

v/hich will also give tho san^ solution as (7 2) p-^ovided a Q 
and H satisry the n cessary conv rgent conditions 7o can 
hov; that the iteration schene (7 23) converg s if 

I \l^ - < 1 

or 0 < a < P/X „ where X is the ma> imum eigenvalue of 
' max max 

QH^H 

If Q = It IS easy to see th^^t the ito ation 

scheme (7 23) converges to the s lution in one iteration if we 

choose a = 2/(X ^ X^,„) where X^,„ is the minimum eigen- 

max mm min 

value of QH^H (vjhon Q = (H^H) ^ ^ ujin ~ 
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It IS clear f om the above discussion that different 
choices Of Q will lead to different iteration schemes For 
g - Im v/e obtain fixed step SOA Since corrputat on o-r the 
best choice for Q which is (H^H) ^ itself is uneconomical 
Ae must choose <4 such that it is a good estimator of 
or the inverse of the channel autocorrelation matrix buch 

choices are difficult to ob am in th samole domain 

Now let us look at the DFT domain representation of 
(7 23) given by 

^(i+l)e = 4e Eie (7 24) 

where as before and is aiven by (7 14) 

IS now an nxn matrix ’"rom (7 2-^) we observe that if 
* 

FQj^F is a diagonal matrix additional computation needed for 
multiplying 1 Qj^F* and ^ is just n multiplications Therefore 
in order to make FQ^F diagonal Qj^ must be a circulant matrix 
Since the best choice for Q is (h 4)'”^ it seems natural to 
choose for whenever is nonsingular This 

feature has been exoloit'^'d by Picchi and Prati [64] in their 
f xst converging self orrhogonalizing DFT domain equalization 
scheme in the context of channel equalization 

So far in this se tion we discussed the application of 
Circulant compiotion for obtaining good estimates of the 
convergence parameter However it may not be always possible 
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to obtain good estimate of the conv rgenco parameter /ath 
Circulant completion In such cases k-Circulant completions 
can be js d For this pu-pose let assum 

that 1 invertible When lk| = 1 it can be shown ttat 
Qj^ IS also a k Circulant Therefore making use of k-DFT 
(7 24) can b written using k Circulant completion of H a 



■^(i+l) ke 

= 4ke 

where 

4 c. 


and 

4ke 

= 


.~lr 


E u = Hu (P.“Hu X 


-1 

Since Qj^ is a k-Circulant matrix Fk'^k^k ^ dia- 

gonal matrix It may b noted that use of k~Cir ulant comple- 
tion r quires computation of k-DFT v;hich can le computed u ing 
FFT algorithm as described in Chapter 5 Th efficiency of the 

iteration scheme depends on tte value of k u co n oirect 
nalysis of th eigenvalue behaviour of Qj^ and consequently its 
effect on a is difficult as it depends upon the channel given 
This aspect needs mor investigation 

7 6 FIXED STEP ST'^hPEST DESCE H ALGORI fHNi 


Th DFT domain version of fixed stco SDA follows from our 
discu S 3 on in Section 7 3 In th s case we can write (7 13) as 


-(i+l)e 


= X 


-IG 


aC 

-le 


(7 25) 


159 


The b St choice of a is qiven by (7 5) as comout t on 
^max ^min ® difficult task stinces of th igon- 

vail s of the channel autocorrelation matrix can be used i\n 
alternative method would be to choose the eigenvalues of -1^ 
or if y are good estimate or the eigervaljes of 

One reason for sue! a choice is that they can b onputed 
easily using DFT //alzman ana Schvarxz [ 1] have used eigen- 
values of in the iterative scheme Foi e /er th y do not 

seem to have noted that a obtained using can also be used 

in the sample domain 

Let and he maximum and minimjn igenvaiues 

of and let 


(7 26) 

max k min k 

If IS c-ose to a th degradation in the convergence rate 

0 th iterative scheme vail not be appreciable Towards this 
end /e prove the following theorem 

Theorem 7 2 Let X^^^ ana b^ the naximum and minimum 

T 

eigenvalues of H H ihen ^rrjgx s ^vax 

^min s - 


where X and X represent maximum and minimum eigen- 

max s min s ^ 

values of where H is a nonsingular square completion of H 

s s s 
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Proof Wc first prove that ||h'^h|| < 
where ( { 1 1 is the spectral norm of a matrix Let P 

be an nxn matrix Using P wo can write 



: 

for l|PH^HgP|l = II 

0 
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Then | jcj ^1 I =1 Therefore 

= M(hX) ^11 l|G“^!i 

Using the result !|ab|| < |j<\|| j|^|| v/e obtain 

I l(H^Hg)“Ml > IKH^Hg) ^ G I 

j h'^h f'^h 

'Jow GHgH = 

0 hJf -hJhCf^h)"^ h’^h^ 

C C Lr C/ 

Therefore I I (Gh3h^)~^| I 2 

o S 

Therefore |)(h'^H)“^1I < | | | | 


Therefore ^ ^ 

min s ■“ mir 

T 

where g and are the miniirum eigenvalues of H^Hg 

T 

and H H respectively v-i 


In particular for k-Circulant completions we have 


^max k - ^max 

and (7 28) 

min k min 

In a aiven situation for fixing « it may be desirable to 
try out different k~Circulant completions and choose the one 
which proviies maximum a But this involves additional compu- 
tations The problem of maximizing a over all possible 
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k“Circulant completions foi a given channel does not appear to 
admit mathematically tractable solutions Thi as ect bovever 
needs further investigation 

In summary we have discussed the role o DFT and k- 
Circulart completions in reducing tie comput tiunal complexity 
and increasing th convergence rate of SDA have shov/ed 

how to obtain better estimates of the convergence parameter 
a using eigen /alues of k-Circulart compl tions of channel 
convolution matrix H As there is a one-to one correspond nee 
between sample domain and DFT domain iteration chemes esti- 
mates of a obtained via k-Circulant completion can be used in 
the sample domain itself to orov de faster convergence 

SDA has only linear converg nee property and tho rate of 
convergence deoends upon the condition number of the channel 
autocorrelation matrix In this connection eif orthogonali- 
zing scheme discussed in Section 7 5 could be useful 

Considering the drawbacks of SDA despite its several 
advantages it appears that when there is severe constraint on 
procos ing time a better method v/ill have to be chosen Th 
method so chosen must converge fast and it must have a better 
rate of convergence Vte wish to note th^t conjugate gradient 
algorithm (CGA) [89] meets thes d mands Howev r it is com 
putationally unattractive because each iteration takes more 
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processing tim compared to SDA Koba/ashi [92] has consi- 
dered CGA for adaptive channel equalization h cr CGA 
docs not app ais to have received much att ntion in the ar a f 
adaptive channel equalization du to its conput tional 
complexity 


CHAPTEP 


O 


1^4 


PERrO MANGE OF DATA REGOVE Y SCHE Lo 

In this chapter v/e give a comp ra^i/e ac oun of oerfor- 
man e of some of the da+a r cover/ schemes tudied in previous 
hapters Perfornance is sjecified in tern o p obabilit/ of 
error (Pe) Sine the pr b bilit> of er ^ oeoends uoon block 
length channel parameters at reco/er/ s hene enplo/cd noise 
statistics and the rate of tran mission over h chann 1 except 
in very special cas s in general it is very diificult to 
obtain close form expressions fo Pe In thi cor nection it 
may be recalled that it as possible to obtain clos form expre- 
ssion for Pe for DR scheme oa iJer d in Chaot r 5 On the 
other hand th schemes consio red in fhaot rs ano 4 are non- 
lin ar in nature and hence are ai ficult to Under 

such situations simulation studies help ir evaluating the 
performance of the Dp sch me erployed Th oughout thi chapter 
we assume binary data transmission Ho ;evcr th analysis can 
also be xt ndod to multil vel du.+a 

This chanter is organ zod in tho following nann r As a 
prelud to the study of th perforroan of various DR schemes 
in Section 8 1 wre charact riio aood c annels for 3DT which 
introduce tsi and provide good performance subject to a given 


1 6 -j 


crit non There ire t /o re sons or kro ang u h channels 
Firstly they provide a benchmark for comparing h perfor 
m nc of other channels condly on coulo u o or f It r 
at the transmitter o as to make the ov rail r soonse of the 
channel nd th p-^efiltcr o resemble do ely a good ch nnei 
Channel examoles used in th simulation stud/ are giv n in 
S ction 8 2 In S ction 8 3 v; scuss performance of BDT 
sy t ns i sing o •‘^formance curv s for a w a range of clock 

1 ng+hs and tyoical chann 1 examples In S ction 8 4 v;c study 

the effect of increasing lol aroig th riv-mbc^s f a block 

v/hich happ rs when the dat rate ov r the ch nn 1 is to be k pt 

the same as source dnt rate 

8 1 GOOD GHAivINELS FOR BLOlK DATA TRAl'o lISoIOi 

811 laximum Mininum Distanc Cnannels 

L t S bo the set of all possible m-l noth d ta vectors 
We know from our discussion in Sec ion 2 4 +hat vhen r is 
received in the pros nc'^ oi /hite Gaussian noise the best 
estimate of the transmitt d vector x is given by a vector 
e such that 

I 1 1 Hx 1 1 < I |r-Hx^ I I for cll e (8 1) 

L t Sy, denote anoth r set dofin a as 

Sy = I Yj - nXj 

I 


^ ^x ^ 


(8 2 ) 


If6 

Arhex V IS the received vector co'»'responding to x ;hen 
noise 1 absent Let d^^ denote f'he distance b tween tv;o 
distinct vectors / and y Then 

= Nil - 1 / J (8 3) 

Du to the presence of noise the deconvolution rule (8 1) 
need not result in x^ which i the actual d ta vector trans- 
mitted In ord r to minim c the o currenc of such situations 

it IS d sirable to have good s paration b ti n any pair of y, s 

3 

in terms of the distance metric given by (8 3) Let the minimum 

distance be d„, ^ 
min 

Vn = "’^" illi-lljll (8 4) 

•^1 ^ Sy 

ifithout loss of gen rality we will consider channel res- 
pons s with unit length A g+1 length vector f is called maxi- 
mum minimum distance { MD) channel sample response if 

= min d^ 1 < 1 3 1 S"' (8 5) 

min ij - 

subject to f^f = 1 and -1 < f^ < 1 0 < i < g xs maximum 

Tho Gcond onstraint in the above formulation is nece- 
ssary + o avoid trivial solutions for example f^ = 1 and fj = 0 
for 3 5^ 1 0 < 1 g 
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d in (8 3) can be written as 

J 

Since conv^olution op ration is coninatativ \i can int r- 
chang the rol s of and h and write 

<3 = (/,-x,) f ( O 

wh re 0 1 ^ 1 7 

and IS the convolution matrix with in he first column 
I t 


it then follow's that the elcm nts of arc ernary 

vanabl s (+2 0 ~2) Moreover ^ is also a onvolution matrix 

^ J 

2 

We can now write d^^ alternatively as 

d?-, = Q-,t I 

wh re IS a symmetric Toeplitz matrix is given by 

^ij “ ^ij ^ij 

In terms of (8 8) the optimization problem (3 5) o com s 

max min (S 9) 

f E 

l<i<2‘^ 

subject to 1) f^f = 1 
and 2) -1 < < 1 0 < i i g 


(8 10 ) 
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Th following lemma provid s -naximum value achi v blc if 
v/e solv (8 9) subject to (8 10) 

Lemma 8 1 In the case of binary alphabet the maximum d 

min 

achievable is at most 2 
p oof Ite make use of the inequ lity 


min ij 




(8 11 ) 


whore ij ^nax ij minimum and maximum eigen- 

values of 

Consider two data vectors which differ in only one posi- 
tion In this case Q is a diagonal m trix with all diagonal 

^ J 

entri s equal to 4 Therefore 


X = X 

run ij max ij 


(8 12 ) 


Using the condition f^^ = 1 in (8 11) we get 


X 

min ij 


< < 


max ij 


(8 13) 


From (8 12) and (8 13) it is clear that w cannot find an f 
for all 1 j such that 


f^Q^jf > 4 for all 1 J 


Using Lemma 8 1 ai prove th fol owing theorem 
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Theorem 8 1 Determination of f which qives d 

*” ^ tnin 

equivalent to solving 

2 4 for all i ^ j and < 4 

f = 1 

“1 < < 1 0 <1 <g 

Proof If f IS chosen s ich that for all of those Q with 

1 j 

^ ^ ^ ^ we have 

min ij 

f^Q f 2 clearly for the res of the Q , s 

i* J J- J 

f_^Qi f > 4 holds from ( 13) 

^ ** 

According to Theorem 8 1 the proc aure for obtaining fVAD 
channels is to first determine all those s for which 
< 4 Then we have to solve the set o^^" dra'f'ic 

min ij 

inequalities given by (8 14) While good amount of literature 
IS available on linear inequalit es [93] very little seems to 
have been acnieved in understanding of systems of quadratic 
inequalities We will not solve this problem directly in this 
thesis Instead wo will consiocr another characterization of 
good chann Is called maxinum distance chann Is (MD-channels) 

/e shalJ see lator that MD—chann 1 haracterization is helpful 
in obtaining flivfiD channels 


- 2 is 


( 14) 
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8 12 naximum Distanc Channels 

We no ; onsider those ch nnel v/hich maximize ^he sum of 
dis met distances between all possible pairs of elements in 
Sy That 1 If tl e pairs a-^cJ give the sam« 

distance = d^^) only one of the pairs will be considered 

in the suiination In this ense the problem ma/ be formulated 
as 


max Z T 

a+1 f Q f 

f eR9+l Q distinct” J” 

J 

subject to (8 10) 

put differently (8 15) can also be written as 

max Q f 

f e 


(8 xo) 


(8 16) 


subject to (8 10) where Q = 2 q (8 17) 

1 j 

distinct 

In the presence of the second condition in (8 iO) the 
problem given by (8 16) is difficult to analy e te therefore 
solve the following simpler problem 

max Q f (8 18) 

f e r9+i 

subject to f’^f = 1 

A solution to problem (8 18) can be obtained by using 
(8 11) from which it follows that the eigenvector corresponding 
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to the maximum eigenvalues is the desireo olJtion to (8 18) 

If this solution satisfies the second constr int in (8 10) then 
a solution to (8 16) is obtained provioed th nininum distance 
achieved is 2 in the binary case Hov^ever 1" proc dure out- 

lined above is computationally unattractiv for large block 
sizes /Moreover it has not been oo siblo to oot in conditions 
under which solution to (8 18) is also a solution to (3 16) 

We feel that a more rigorous study nay lead to interesting 
result In the absence of such r suits v/e have follov/od the 
procedure outlined belov^ for obtaining \D as tell as MD 
channels 

8 13 Procedure for Obtaining WiD and ID Channels 

1 Given the block length m and number of ISI samples g 
form the Q matrix given by (8 17) 

2 Determino the normal! z d eigenvectors and the correspond 
ing eigenvalues of Q 

3 The eigenvector corresponding to the maximum eigen- 
value IS a solution to (8 18) 

4 Compute the minimum distance for eacn eigenvector satis- 
fying the second constraint in (8 10) 

5 The eigenvector giving maximum ninimum distance is^ a 
solution to (3 14) if the riaximum minimum distance is 
2 in the binary case 
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It must be noted that le may not g t solutior to (J 14) 
as given by the last step in all the cases To ;ards this nd 
Afe conducted sone nanerical experiments to ootain channel 
r ponses using the above procedur for s ver 1 values of n 
and g Some of th s" results are pr seated in Table 8 1 In 
the last colurin of the table we have given the mininum dis- 
tances achieved In all th'^ cases ; h je obtained both fA\D 
as well as MD channels Another int restin feature is that 
son of the channel responses are reminiscent of the 
p rtial response channels [94] For instance in the case of 
two ISI terms the channel responses ( 707 0 707) and 

(- 707 0 - 707) are duobinary channel responses The channels 
obtained for g = 4 5 6 also show similar features 

The foregoing observation is interesting because partial 
response signalling schemes were or ginally sujgested in order 
to achieve better spectral shaping of the transmitted signal 
As we do not have yet a formal definition of such schemes it 
IS difficult to establish meaningful relation between spectral 
shaping and maximum distance separation This aspect needs 
further inv stigation ^t this ooint we wish to note that 
Folconor and f«/lagoe [47] also have shown that optimum desired 
impulse of th channel and prefiltcr combination of long h 
two under minimum mean square crit non must b the duobinary 
xmpulso response 
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futD channels with = 2 have another interesting 

property when used in conjunction with L-d convolution or 
Viterbi algorithm For such channels there v;ill bo essentially 
no loss in SNR when ML-dc convolution m thod is adopted This 
conclusion is based on the analysis given by Forney [38] H 
has shovjn that in the cas of naximun lik lihood sequence 
estimation the symbol error probabilit/ is bound d by 

Kl < Ku Qtdn^n/Ho) (8 W) 

where 

Q(x) = 7 cly (8 20) 

Y^2it X 

and and Ky are constants independent of noise variance and 
the channel parameters K, is typically vathin an order of 
magnitude of V/e kno\/ that for one shot transmission (no 

ISI) the probability of error is given by 

Pe(one shot) = Q(2||f}|/NQ) (8 21) 

Thus if d - 2 1 If I I the probability of error achieved in the 
presence of ISI using ML-*dec on volution procedure will be appro- 
ximately equal to Pe (one shot) 

8 14 Computation of Minimum Distance of a Channel 

A direct method of d ternining ^.s to compute the 

distanc between all possible pairs of s belonging to set Sy 
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given by (8 2) This is a JifficJlt j Db even for noderatf' 
block lengths We now show that J/L-POA d v 1 d ii Chapter 3 
can be modified for conpiting 

Fr m our earlier discussion in this chapt r it is clear 
that the problem of finding ninimum distaa o cun be formulated 
as 

min (x -X 

_ J ^ J 

X X € S 
1 J X 

Clements of z ar^- ternary variables taking vnlues from 
the set {-2 0 2} Let z denote a v tor whose el ’n nts are 
ternary variables In terms of z the ibove probl r can be 
alternately formulated as 

min z (8 22 ) 

z 

subject to z^z > 0 (8 23) 

Note that the constraint zz > 0 is ncces ary to eliminate 
all zero solution for cas s corr spending to = x^ 

The foregoing problem is a quadratic optimization problem 
in the presence of a constraint In order to find the non- 
zero minimum value we adopt the procedure of Chapter 3 with the 
following difference In the fir t step w^oh z-^ is optimized 
for Gv ry given value of Z 2 Z 3 ^g +1 only non zero 
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s Re t of the recursion stage are c rried out \/i i out any 
constraint This follows from the fact that s nee Zj^ is con- 
strained to take only nonzero /alues the constraint (8 23) is 
satisfied and hence rest of xhe s can be hosen arbitrarily 
vVe wish to note that in the context of CSDT Burkhart and 
Barbosa [95] have sugges ed a nodified Vit°rbi alaorithm for 
determining the minimum dis ance of a gi/en bannel 

8 2 CHAItsIwLS USED IN THE STUDY 

In order to compare the performance of diff rent data 
recovery schemes it is necessary to consider channels \-?hich 
show both merits as well as dements of various DR schemes 
The channel examoles considered are given in Table 8 2 In 
iaole 8 2 the first two channels A1 and A2 are maximum dis- 
tortion channels considered oy Austin [13] The next two 

Table 8 2 Chann Is used in the simulation s Jd/ 


Channel sampled response ^min 


A1 

3-V2(i 

1 1) 

1 

633 

A2 


1111) 

1 

265 

Cl 

2-1/2 (. 

■ 235 667 1 667 - 235) 

1 

933 

C2 

2“1/2 ^ 

235 667 1 667 235) 

1 

188 

Pi 

( 227 

460 688 460 227) 

1 

131 

P2 

( 407 

815 407) 

1 

630 
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exanple<3 Cl and C2 taken from [4] Th 1 st tv;o exa les 
Pi and P2 are taken from [7] 

Maximum distortion channels as the name it elf indicates 
exhibit maximum realizable disto tion according to some distor- 
tion measure Austin [13] determined such channels b/ choosing 
the samples of the channel autocorrelation co ffici nts 
a(-g) a(0) a(g) so as to maximize 

2 |a(i)| 

i=-g 


subject to 1 a(0) is unity 

g 

and 2 A{w) = E a{i) oxp -jiw) 

i=-g 



The second condition implies hat the channel is realiza- 
ble Austin has shown that the sairpled auto orrelation fun- 
1 ction having a triangular envelope corresoonds to maximum 
I distortion channels in the sense defined above 

i 

8 3 SIMULATION STUDIES ON DATA R COVER\ SCHfcfiES 
^ The following data recov ry schemes are considered 


'I 

i 


1 





ML data recovery scheme for CSDT 
ML data recovery scheme for BDT 


} LC 
MLB 


2 
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3 Decision feedba k scheme for CSDT DFC 

4 Modified Austin s appioach for BDT fiA3 

5 Fourier expansion method with decision 

leedback P £3 

6 LS data recov ry scheme for BDT LSB 

7 Circulant completion ba oc, cher e for BDT CGB 

8 Skev/ Circulant completioi bas d ch me for 

BDT oCB 


Note that a three letter idontiiier is associ t d vath each 
data recovery scheme these identifi rs have been used through- 
out in the text tables and figures Wo uish to note that LSB 
IS based on Fourier expansion method without decision f edback 
In all the simulations 10000 data s/mools taxing binary (-^1 1) 

values were used The curves plotted arc obtained using single 
simulatioi 

In Figures 8 1 - 8 9 w have shown the variation of th 
probability of error with SNR for the six channels given in 
Table 8 2 Performance curves in th case of MLC are obtained 
Using ^U,-RDA given in Section 3 2 vath tho following 

modification D cision regarding a transmitted data is made 
after a fixed no of recursions say ID That is aft x the IDth 
stage we follow the path corresponding to the maximum value of 
the objective value achieved so for nd chooso the data value 
oxresponding to first stage After tha*t wo shift the ID stage 
results ono stage bacx so that the second stage now becomes the 
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flG 8 2 Comparison of performance of various OR schemes 
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first stage The procedure is continued till all the trans- 
mitted data are decided The value of ID i m it oied in the 
figures The decision feedback scheme for C DT (DFC) is based 
on the eauations given in [?] The number or for\;ard taps used 
is den ted by ^T in the figures For Ih ake of co pari on error 
probability curve for channels vath no ISI is Iso shovm in 
the figures 

The Figures a 1 to '^9 cover a sufficiently wide rang or 
block lengths and channel responses The figures indicate that 
the performance of MLB is better than MLC \;hcn ID is equal to 
the block length In the course of the simulation studies we 

have observed that for obtaining good performance at medium and 
large S’MR values ID must be large in the case of PLC In 
Figure 8 10 we have compared the performance of MLB and /ILC on 
th ma'<imum distortion channel (1 1 1 1 1) Figure 8 10 shows 
that ILC performance saturates if ID is not la g 

From the Figur s 8 1 - 8 9 we also observe that MAB performs 
well compared to DFC when the block length is equal to the numb-^r 
of forwaid taps However it must be kept in mind that MAB 
requir s more memory conpar d to DFC It may b recalled that 
the computational complexity of both he schemes are more or 
less same FEB how ver does not p rform as well as MAB One 
reason is that FlB assumes no knowledge of the noise variance 
One attractive feature of FEB is that it can handle illconditio— 
ned channels more efficiently compared to the Trench algorithm 
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based MAB in which illconditionea ch nnels can j r se to 
large values of the tap coefficien s 

It IS interesting to note that in the case of Cl (see 

Table 8 2) there is essentiaJly no differ nc oet^een fALC 

and fnLB for block length of 8 12 16 In fac rc peifornance 

IS also very close to MLB One reason for t is perhaps is 

that d for this channel is 1 93 i/hereas for the other 
rnin 

channels it i much less than 2 (see Table 8 2) In order to 
check the validity of this claim in Figure 8 11 we have 
plotted the performance curves fo the example given in [33] 

For this channel is 2 Performance curves indicate that 

the loss in performance due to CCB conpared to vlLB is not 
ignificant This observation however needs more inv stxga- 

tion 

Figure 8 12 shows the effect of block length on the per- 
formance of CCB and SOB in the case of channels Cl and PI 
Although the performance in the case of channel Cl 
is better compared to that with PI ~ ^ perfor 

mancG depends strongly on block 1 ngth 

8 3 1 The Effect of Increased ISI Amono the Block Elements 

So far we have considered BDT in which the efrective data 

rate over the channel is less than the source data rate due to 

, , ^ hi nrk H now consider the situation in 

g zeros following each blocK « 

1 = block are txansmitt d at a higher 

which the data values of a blocK are 




FIG 8 11 MMD Channel performance 
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FIG 8 12 Effect of m on the performance of CCB and SCB 
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rate so that there is no reduction m +h rj 4. 4.^ 

iwuu^xion in th a r t ovpt the 

channel However +his j ^ 

IS results in increased lol among 

block members The effpr-i nf 4.^ . . 

c or incr asing tho data rate on the 

performance is the topic of this section 

First we consider maximum distortion channels In 
Figure 8 13 we have shown the effect of increasing ISI /hen 
MLB IS employed For the sake of comparison /e have also 
plotted MLC performance for ID equal to the bloci length Note 
that the MLC curve for ID = 16 aturates aft 14 dB 

Figure 8 14 shows the perforn ncc curves in tho ca e f 
DFC and MAB In this case also we obs rve that thcro is 

improvement in the performance of MAB compared to DFC 

As another example we considered a telephone channel 
given in [33] In Table 8 3 we have given the equivnlont 


Table 8 3 Channel sampled response coefficients 


Channel 

Symbol 
duration 
(in nano- 
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FIG 8 13 Effect of increased ISI on the performance 
of MLB 
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FIG 8 14 Effect of increased ISI on the performance 
of MAB 
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discrete time response values for various data ra es Thp 

performance curv s are shown in Figures 8 1 J 8 16 In 

this case however BDT does not seem to enjoy noticeable 
advantage over CSDf 

Summarizing w evaluated th performance ot various date 
recovery schemes using simulation It may be noted that c rc 
must be exercised while making observations or ihile drav/ing 
conclusions regarding a particular sch nc bas q on simulation 
results Thus with limited numb r of 'Examples and in the 
absence of any theoretical justification it is difficult to 
mate any definite conclusion regarding the upenority of 
one schem over the other Howevo'*' from our limited simula- 
tion experiments on typical channels it app ars that by 
choosing moderate block size and with incr ased data ra e 
over the chann 1 LB can be effectively implemented at a 
reasonable computational and storage conole co o red to 
MLC which requires large amount of storage as ID >> 5g 
Similarly we have obs rved that when the value of d^^^^ is 
close to 2 COB or SCB works w 11 compared to MLB with 
relatively small amount of SNR loss VAB also seems to be 
effective compared to DFC although it requires more storage 
yoreover it limits error propagation to a block length in 
the worst situation - a feature absent in the case o- DFC 
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FIG 8 16 Effect of increased ISI 
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CHAPTER 9 

COHCLUSIOHS AND SUGGESTIONS TOP FUPT-£R P£SEARCr^ 


In this thesis we have investiaated the problem of 
recovering data in block data transmi sion systems We have 
develoo d several data recovery schemes and com^ ared their 
performance with DR schemes available for continuous serial 
data transmission s stems The data recovery proolem has been 
viewed as a finite deconvolution probT m in th<= oresence of 
noise Three deconvolution methods have bo n studied in 
detail They are maximum likelihood decision feedbacl and 
least ouares deconvolution methods Tho DR schemes developed 
on the basis of these deconvolution methods may be vie/;ed as 
counterparts of the maximum likelihood decision feedback and 
linear DR schemes in the context of CSDT 


At this stage it may be appropriate to note the following 
differenco between BDT and CSDT systems Due to practical 
consioeratlons in CSDT systems truncated number of ooserva- 
tions are used for making decision r garding a transmitted date 
symbol although in theory vr must use the entire observation 
values Thus the numb r of observations used for making deci- 
sion determines the P rfcrmance of a given DR scheme In BDT 
we control the data rate and data transmission format so as to 
achiovo bettor trado-off between error performance and oonputa- 

tional complexity 
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Sail nt features o'" the study earned out in this thesis 
and sugges ions for further research ar gii/cn in th following 
para:jraphs 

The mayimum likelihood recursiv d c nvolution al rithm 
dovolopcd in Chaptei 3 is simpl to understand compared to the 
convontiona Vitcrbi aljoiithm approach v/hich m-''cs use of 
Bellman s principle of optimality and state r p'rcs'^n^'ation The 
ML~RDA developed u ing Mit en s 1 m la provi los nor insignt 
into the data recovery problem compar o to tt Vits^rbi algo- 
rithrr application The cornoutational cornpl xity and storage 
requirements of ML RDA grows linearJy 'ixth the block lengtn 
and exponentially with he channel memory Simulation studio 
on some typicil channels using iL RD for BDT and CoDT s /stems 
show thnt ML-RDA for CoDT system requires more storage and 
computational complexity compared to ML-RDA for BDT systems in 
order to achieve same error performance 

W have further shown that in the case of birary dat^ it 
IS possible to reduce both storage and computatior al require- 
ments of ML RDA by using results from the theory of Boolean 
and pseudo-Boolean functions Dcveloom-^nt of similar methods 
for multil vel data using some of ne t hniou s given in [72] 
for dealing with functions of disc^ to variabL s i an 
interesting area for further research 
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Ue have also shown that the method of minimizing pseudo 
Boolean functions provides a goo frameuor^ ^or developing 
ML~decoding sch mes for binar/ error correcting block codes 
Examples given in Ch pier 3 illustrate the usefuin ss of th 
method described h however have not stucied this aspect 
in detail The technique d velooed in this thesis can be 
extended to nonbinary codes al o aetailed study of codes 
over other fields using sone of the r suits available in [72] 

IS expected to prove usefu in a better understanding of ML 
decoding of linear as well as nonlinear block cooes 

The decision feedback sche'nes developed in Chapter 4 are 
computationally more attractive in s tuations ‘vhere ML-RDA 
becomes uneconomical Unlike the conventional methods of 
deriving decision feedback data recovery schemes w have moti- 
vated the idea of decision f edback via Gaussian elimination 
method ^e have shown that th moaifiod Austin approach can 
also be viewed as Gaussian eliminatior method with a nonlinear 

decision device 

Tho DF-DR sohotnes for BDT systems restrict error propaga- 
tion to atmost a block length whereas in the cas of OF-DP 
schemes for CSDT systers error propagation can extend to much 
longer durations Hcv,ever DR sch mas f r .DT systems require 
mors storage cotnparedro DR schoncs for CSDT systems for 
achiev ng same probability of error performance 
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The DR scheme studied in Chapter 5 have the lea t 
computational and storage requirements compared to t cse 
studied in Chapters 3 and 4 However thi i only at t e 
exoense of pr-obabilicy of error performer ce Hov;ever simu 
lation studies snow that ir the case of a gooa Tunb<=r or 
channels DR schemes based on V Pirculant completions per orms 
well compared to DR schemes based on LS-deconvolution ie 
have shown that k Circulant completions of th channel convo 
lution matrix provides a more gen ral etting for studying 
deconvolution problems using di cr te Fourier transform rt 
detailed study of optimum choice of 'c-Pirculant completions 
according to some performance criteria needs further investi- 
gation 

In Chapter 6 i/e have showed hat deconvolution using DFT 
and least squares deconvolution can be viewed as particular 
cases of square completions of the channel convolution atrix 

Detailed anal/ is of he Steepest d scent algorithm in 
Chapter 7 shows that in the case of soir^ examoles good estima- 
tes of the convergence parameter can be obtained using 1 - 
Circulant completions This analysis clearly explains the 
observation made by others that DFT domain fixed tep steepest 
descent algorithm converges faster han the sa pie domain 
chemes Moreover w have shown that such cstimar s can be 
used in the sample domain 3 achiev go.d conv.roence 
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The simulation results given in Chapter 8 cov "a /il 
range of block lengths and typxc 1 channel exa pi s It is 
interesting to note that the offec of incr as'^d ISI among 
the symbols of a block duo to increased data rate ov r the 
channel on the perforrance is not signiiicant Sirrulation 
studies also indicate that vith appropriate ct icc of block 
lengths BDT provides better tx dc“ 0 ff b tv/een rror rforrrance 
and processing c rplexity corrpar d to th av il blc DR schemes 
for GSDT syste is Howev r as imulation results arc alvja/s 
based on a few examples n definite claims can be made regard- 
ing the superiuri y of BDT ov r CSDT 

The maximum minimum distance and maximum dist nco channels 
obtain d in Chaotex 8 shoi^ th t seme of tho good ctcinncls are 
partial responso channels How ver in tho absence of any clear 
definition of partial response channel [94] it is difficult to 
show the conncceion between paitial rtsoonse no MAD or MD 
channels This asp ct needs further investigation 

Another asoect of tie data r covery problem in vhich we 
could naz achieve much success is in obtaining go c bounds for 
the error p rfrrm nee of max mum likelihood ana d cision f d- 
back sch mes The difficulty socris to be du to the nonlinear 

nature of the DR schemes However w still f ol that it nust 
b possible to obtain useful bounds for orobability of error 
which in this case will bo functions of block length and channel 
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menory ihi aspect requir s further in /estigation Some 
v\/ork in this direction has b en carried out bv Klein and 
J^olf [ 54 ] 

In his thesis we have not considered the problem of 
adapting the parameters of th data recovery schem s to the 
channel variations Depending on the nature of the channel 
variation the desion of th dat r covery schemes v/ill change 
The problem is interesting due to th presence of noise and our 
limitation in recovering data vatnout errors Further investi- 
gation is needed to understand various aspects of this problem 
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APPeroi/ A 
BITTEN S LE \A 

In this appendix we state and orove Mitten lernina [55] 

Mitten s Lemma Let X = (x) be an arbitrary set and let 
Y^(x; = (y^) 1 = 12 n be finite seqaence o arbitrary 

sets whose definitions may depend on y £ X Let y^^) 

1-12 n be a sequence of real /alued functions for 

which g. (x) = min[g (x y, ) lY, e Y, (x)3 exists for i = 1 2 n 

*L JL Ju J* X 

and all x e Let f(x) be an arbitrary function and let 
g(x Yi y^) - h[f(x) gj^(x vj^) gi^(x y^^)] be 3 real 

valued function for which g = min[g(Xj^ y^^ y^^) j 

X e X y, e Y. (x) 1 = 1 2 n] ists and h u) = h[f (x) 

g^(x) g^(x)j exist for all <eX If 

h (x) < g(xjL Yi /„) 

1=12 n then g = min[h^ (x) |x e X] = h 

Proof The lemma can b proved the method of contradiction 
For this purpose assum that there exists x e X and y^£Y^(x ) 

1 = 12 n such that g(x y^ Y^) < 

But from the definition of minimum h < h (x ) and 
h (x ) < g(x Yi Yp) assumed property of h U) 

Therefore 

g(y y^) < h ^ ^ ^ '^n^ 


which IS a contradiction 
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APP r DIX d 
k CIRCUL n UTRICt 


The puroose of this apoen ix i to list a re\/ properties 
of k-Circuiani matrices [8i] [82J 

The k-Circulant matrices con adored in oir study have the 
following structure 



where jk) =1 When k = 1 is a Circulant natrix k = ~1 

gives skew Circulant matrix Skew Circulants ar also known as 

negacyclic matrices [82] 

It IS convenient to stuay k-Circulant /itn xhe help of 
the simple k-Circulan matrix 
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\ matrix A is said to be k Circulant iff 

in terns of Cj^ can be expressed as 

S nee every k Girculant latrix commutes vath r). ir is suffi- 
cient to know the eigenvalu s and eig nvectors of in order 
to characterise and analys Circulants 

Ihe eigenvalu s of are the nth root of k for the 
characteristic equation of t)j^ is given b/ 

~ k = O 

Lot p be an nth root of k Th p i = O 1 n-1 

Jhere jj = represents n roots of X^-k = 0 

We now show that th v ctor 

X = (1 x“^ x“^ j^”(n“-l)) ^ 

root of k IS a eigenvector of Ttj^ 

A simple calculation shows that 

ly-(o-l). 

1 

n,; X = , *.2 “ ^ 

( n*~*2) 

X 
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Thus n roots of k generate n eigenvectors of r|j Let F,^'^ be a 
matrix with the n eigenv ctors of as columns It has the 


following 

form 





1 

' 1 

1 

1 

j 



p-^ 


(P^" h 

1 


n-l i 






= TK 

p-^ 

(poo) ^ 

(pu." h 

2 

iu 2) 




(p^"'h' 

•(n-l) 


Note that ^ is just a normalization 

factor used 

for making 

'=k = In 





Using B 2 it can 

b shown that 






= % 



(B 3) 

where A ^ 

IS a diagonal matrix with pw^ 

1=01 

n- 

1 as 


d agonal elements 

Wh n k = 1 ^k ~ ^1 ” ^ matrix is obtained In 

virtue of this wo shall refer to as k~DFT m trix In t rms 

of F we can write Fj^^ a 


-1 “1 ~1 

Fj^-^ « D 


(B 4) 


where D is a diagonal matrix with i == 0 1 n-1 as 

diagonal entries Using (6 4) in (B 3) v obtain 
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F u ="l< 5) 

If 0^ IS a Circulant ma rix we kno ; hat 

r (ti 6) 

where is a diagonal natrix with eigcnv lu s of as 
diagonal entries 

Comparing (B 6) ano (d 5) we see that j D ^ repre- 
sents a Circulant matrix 

Post and pe multiplying 8 1 ana re pectiv ly 

and using (B 3) \e obtain 


" V ^ ^1 ^'k + S 


C A 
n-1 ^ k 


(B 7) 


From (B 7) we see that if ^ 3 ^ i=Ol nl are the 

eigenvalues of Cj^ 

X = ^ 2 } C (pw^)^ 8) 

1 j=o J 


(B 8) can also be rewritten as 

X = (C pJ) 0)^^ (3 9) 

^ J=0 ^ 

(B 9) exhibits an interesting aspect eigenvalues of Cj^ can be 
determined by finding the eigenvalues of a Circulant matrix 
having firs column 

(Co Cj^P C2P C„_^o ) 


(B 10) 
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This can also be seen directly u ing (B 7) Usina (d 4) in 
(B 7) we get 

FDC^D-^r^ CBll) 

Therefore ^ is a Circulant matrix B/ little calculation 

we can see that the first column of this n trix is given by 
(B 10) 

Finally consider inverse of a l-Circulant riting (B 3) 
in terms of r|~^ whenever exists /e get 

n ^ 

« " k^ 

which shows that inverse of a k Circulant whenever it exists 
is also a k-Circulant 
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